-- Logs begin at Thu 2022-07-28 05:21:45 UTC, end at Thu 2022-07-28 06:47:49 UTC. -- Jul 28 06:08:06.293845 dv-cinder-812429-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 06:08:06.301189 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 06:08:06 2022] *** Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: nodename: dv-cinder-812429-2-iscsi Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: machine: x86_64 Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: clock source: unix Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: pcre jit disabled Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: detected number of CPU cores: 4 Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: current working directory: / Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: detected binary path: /usr/bin/uwsgi-core Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: your processes number limit is 47680 Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: your memory page size is 4096 bytes Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: detected max file descriptor number: 2048 Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: lock engine: pthread robust mutexes Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: thunder lock: enabled Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 06:08:06.305129 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 06:08:06.344615 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: Python main interpreter initialized at 0x5577309ca9c0 Jul 28 06:08:06.344615 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: python threads support enabled Jul 28 06:08:06.344615 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: your server socket listen backlog is limited to 100 connections Jul 28 06:08:06.344615 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: your mercy for graceful operations on workers is 90 seconds Jul 28 06:08:06.345006 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 06:08:06.345072 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: *** Operational MODE: preforking *** Jul 28 06:08:06.345368 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: *** uWSGI is running in multiple interpreter mode *** Jul 28 06:08:06.345368 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: spawned uWSGI master process (pid: 80308) Jul 28 06:08:06.345580 dv-cinder-812429-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 06:08:06.348639 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: spawned uWSGI worker 1 (pid: 80317, cores: 1) Jul 28 06:08:06.348639 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: spawned uWSGI worker 2 (pid: 80318, cores: 1) Jul 28 06:08:06.348639 dv-cinder-812429-2-iscsi devstack@keystone.service[80308]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 06:08:11.983471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 06:08:11.983471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: from cryptography.utils import int_from_bytes Jul 28 06:08:11.984277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 06:08:11.984277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: from cryptography.utils import int_from_bytes Jul 28 06:08:11.984698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 06:08:11.984698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: from cryptography.utils import int_from_bytes Jul 28 06:08:11.985157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 06:08:11.985157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: from cryptography.utils import int_from_bytes Jul 28 06:08:12.034588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 06:08:12.034773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 06:08:12.034967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] command line args: [] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 06:08:12.035064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 06:08:12.035170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 06:08:12.035260 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 06:08:12.035367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] command line args: [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 06:08:12.035450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ================================================================================ {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 06:08:12.035650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 06:08:12.035738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] admin_token = **** {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.035818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ================================================================================ {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 06:08:12.035894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] config_dir = [] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.036075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] admin_token = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.036157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.036258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] config_source = [] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.036335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] config_dir = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.036444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] debug = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.036527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.036647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.036965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] config_source = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.036965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] debug = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.037157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.037277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.037277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.037502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.037502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.037696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.037793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.037892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.037986 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.038088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] list_limit = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.038088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] log_config_append = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.038277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.038378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.038486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_config_append = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.038579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] log_dir = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.038672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.038809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] log_file = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.038919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_dir = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.038919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_file = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.039614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.039749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.039868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.039947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.040090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.040239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.040327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.040508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.040595 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.040678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.040755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.040871 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.040949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.041123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.041209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.041399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.041489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.041637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.041712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.041874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.041951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.042111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.042234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.042348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.042430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.042597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.042679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.042847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.042969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.043125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.043365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.043550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] publish_errors = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.043628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.043785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.043872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] publish_errors = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.044017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.044095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.044207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.044280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.044425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.044511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.044660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.044744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.044896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] standard_threads = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.044973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.045129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.045207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] standard_threads = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.045385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.045477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.045550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.045618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.045729 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] use_journal = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.045809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.045906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] use_json = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.045982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] use_journal = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.046083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] use_stderr = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.046159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] use_json = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.046303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] use_syslog = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.046375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] use_stderr = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.046448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] use_syslog = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.046517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.046683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:08:12.046760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.046946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.047024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.047189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.047267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.047431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.047508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.047685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.047762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.047923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.048000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.048167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.048247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.048404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.048481 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.048649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.048731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.048896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.048973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.049147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.049224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.049403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.049492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.049652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.049734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.049889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.049969 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.050133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.050208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.050380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.050463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.050621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.050698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.050865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.050940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.051123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.051205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.051372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.051450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.051611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.051694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.051849 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.051998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.052136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.052216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.052361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.052439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.052609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.052694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.052844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.052921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.053079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.053155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.053322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.053411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.053571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.053649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.053813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.053889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.054063 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.054146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.054320 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.054433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.054570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.054732 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.054835 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.055049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.055118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.055254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.055321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.055676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.055759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.055825 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.055934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.056030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.056123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.056246 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.056350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.056453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.056555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.056671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.056775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.056903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.057034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.057120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.057235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.057381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.057502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.057623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.057724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.057864 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.057953 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.058105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.058184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.058341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.058413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.058579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.058664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.058817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.058893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.059063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.059140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.059299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.059387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.059541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.059620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.059776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.059856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.060016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.060104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.060255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.060339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.060494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.060577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.060729 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.060815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.060976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.061052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.061212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.061286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.061475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.061557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.061711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.061790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.061952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.062030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.062191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.062276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.062424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.062501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.062664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.062741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.062902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.062987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.063141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.063217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.063384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.063460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.063641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.063793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.063889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.063966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.064124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.064200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.064363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.064449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.064600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.064681 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.064840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.064921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.065077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.065165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.065335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.065417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.065581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.065658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.065820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.065907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.066059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.066138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.066295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.066374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.066534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.066610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.066772 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.066849 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.067013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.067096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.067256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.067340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.067499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.067577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.067744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.067828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.067992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.068068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.068228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.068306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.068466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.068543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.068703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.068785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.068936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.069021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.069170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.069247 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.069426 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.069505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.069665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.069744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.069932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.070036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.070164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.070341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.070423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.070500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.070635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.070712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.070883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.070961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.071138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.071217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.071381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.071459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.071643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.071720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.071888 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.external = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.071966 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.072124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.072202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.072384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] auth.external = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.072461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.072623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.072727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.072867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.password = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.072946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.073102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] auth.token = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.073180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.073366 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.073449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] auth.password = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.073609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.073689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] auth.token = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.073854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.073938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.074088 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.074171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.074340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.074423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.074577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.074660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.074811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.074894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.075051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.075130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.075296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.075375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.075543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] credential.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.075623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.075793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.075871 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.076034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.076112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.076281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.076360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.076520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.076600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.076769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.076845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.077010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.077096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.077263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.077355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.077549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.077628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.077792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.077951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.078052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.078130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.078331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.078409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.078579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.078665 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.078840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.078920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.079089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.079167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.079332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.079418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.079585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.079664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] federation.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.079830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.079906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.080064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.080149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.080317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.080396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.080552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.080628 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.080807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.080894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.081049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.081134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.081338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.081419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.081570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.081660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.081833 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.081910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.082077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.082155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.082329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.082415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.082572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.082649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.082811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.082888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.083052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.083138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.083299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.083382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.083535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.083612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.083778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.083864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.084017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.084093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.084261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.084338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.084497 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.084586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.084757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.084873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.084993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.085071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.085231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.085345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.085493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.086085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.086259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.086344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.086515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.086604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.086771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.086852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.087032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.087117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.087287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.087368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.087658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.087753 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.087826 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.087896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.088035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.088123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.088270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.088348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.088508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.088617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.088743 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.088824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.088983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.089062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.089228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.089324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.089498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.089580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.089750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.089827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.089985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.090062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.090229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.090308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.090480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.090560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.090719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.090796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.090955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.091034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.091197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.091279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.091431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.091499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.091671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.091843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.091921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.091985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.092136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.092216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.092377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.092456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.092618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.092723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.092850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.092960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.093090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.093208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.093433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.093566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.093566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.093753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.093878 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.094009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.094089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.094163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.094281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.094360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.094522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.094603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.094763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.094841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.094998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.095079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.095237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.095316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.095476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.095554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.095721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.095798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.095971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.096051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.096209 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.096287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.096459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.096538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.096698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.096779 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.096933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.097012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.097174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.097253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.097430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.097511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.097672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.097751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.097914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.097993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.098155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.098237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.098391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.098470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.098644 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.098722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.098897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.098977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.099152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.099230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.099391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.099530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.099641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.099788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.099867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.100029 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.100107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.100283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.100369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.100527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.100684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.100762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.100940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.101019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.101218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.101911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.102020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.102090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.102300 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.102391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.102545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.102621 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.102759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.102839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.102999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.103086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.103240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.103319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.103482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.103559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.103725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.103811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.103971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.104049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.104205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.104282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.104446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.104532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.104689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.104774 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.104933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.105012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.105179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] resource.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.105265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.105453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.105533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.105707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.105785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.105950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.106039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.106190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.106268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.106442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.106521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.106683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.106770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.106925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.107005 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.107165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.107243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] role.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.107411 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.107497 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] role.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.107645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.107724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] role.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.107896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] role.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.107973 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.108133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.108220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.108386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.108466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.108636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.108721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.108887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.108976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.109138 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.109218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.109412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.109496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.109664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.109755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.109902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.109981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.110153 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.110232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.110392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.110477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.110645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.110724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.110897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.110975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.111150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.111319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.111399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.111559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.111809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.111889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.112054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.112140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.112304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.112382 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.112557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.112636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.112797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.112876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.113051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.113130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.113290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.113431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.113561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.113641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.113795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.113874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.114029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.114108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.114264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.114345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.114505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.114584 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.114745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.114824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.115003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.115084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.115242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.115320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.115481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.115559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.115715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.115796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.115954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.116032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.116194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.116271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.116437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] token.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.116517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.116674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.116751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.117662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.117740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.117827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.117890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.117947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.118004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.118187 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.118267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.118431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.118511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.118678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.118756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.118916 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.119000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.119159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.119241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.119396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.119476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.119638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.119716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.119879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.119959 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.120110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.120177 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.120355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.120435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.120617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.120701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.120844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.120922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.connection = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.121134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.121222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.121411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.121510 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.connection = **** {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.121653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.121741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.121899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.121987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.122171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.122264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.122415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.122490 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.122603 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.122683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.122841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.122920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.123080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.123160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.123316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.123383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.123561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.123640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.123800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.123885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.124048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.124127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.124364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.124530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.124613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.124764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.124843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.125007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.125085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.125242 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.125362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.125438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80318) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 06:08:12.125509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.125743 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:08:12.125917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80317) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 06:08:12.155402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.155579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.155674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.155754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.155831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.155931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.164326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.164432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.195631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.195958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.196407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.196648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.255133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.255497 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.307763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.307999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.308213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.308455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.374675 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.374905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.375018 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.375129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.375218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.375358 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.375484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.375582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.375663 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.375786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.375869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.376005 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.399719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.399953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.400043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.400192 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.400277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.400421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.400514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.400642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.400725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.400859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.400943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.401073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.401159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.401293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.412525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.412780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.425898 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.426116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.426253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.426341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.426537 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.426796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.426896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.426974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.427048 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.427121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.427193 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.427274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80318) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.427350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.427556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80317) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:08:12.444665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.444802 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.444914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 06:08:12.445535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 06:08:12.445752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.445915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.445992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 06:08:12.446065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 06:08:12.446209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.446295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.446470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.446553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.446725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.446863 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.446992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.447074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.456758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.456866 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.457032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.457130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.457324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.457410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.457612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.457695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.465107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.465225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.469934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.470045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.470216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.470301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.470503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.470588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.470786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.470871 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.471071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.471163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.471332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.471415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.471599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.471683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.471876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.471962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.472153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.472235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.472431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.472512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.472697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.472780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.495829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.496012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.497330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.497536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.497622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.497822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.502284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.502476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.502573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.502763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.502847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.503055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.510143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.510298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.510397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.510585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.514620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.514804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.514920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.515095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.515206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.515391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.515474 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.515679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.515770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.515959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.516042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.516254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.516335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.516530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.516620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.517065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.533349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.533629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.533762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.534045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.537608 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.538007 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.541418 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.541717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.541899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.542022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.546180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.546464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.549370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.549688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.552634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.552872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.552960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.553176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.553271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.553495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.553589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.553779 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.553873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.554072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.554158 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.554954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.571661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.571783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.571945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.572031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.572218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.572302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.572492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.572585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.579415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.579516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.580985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.581084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.581248 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.581382 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.584072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.584213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.584355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.584496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.584659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.584793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.584945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.585087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.585235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.585398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.597421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.597589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.597705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.597875 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.598002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.598156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.598243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.598438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.598524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.598755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.598850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.599001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.599084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.599293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.614873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.615046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.618044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.618132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.620995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.621080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.623550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.623654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.625085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.625195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.625406 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.625520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.625694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.625781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.632177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.632282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.634525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.634676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.637499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.637599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.637796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.637894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.638088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.638172 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.638368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.638448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.646746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.646847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.647035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.647120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.647318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.647399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.654881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80318) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.654980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80317) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:08:12.655164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.655248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.655460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.655561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.655738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.655823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.656027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.656110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.656304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.656396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.656576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.656660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.656857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.656938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.657228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.657341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.657526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.657609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.657820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.657913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.658090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.658173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.658354 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.658435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.658617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80318) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.658702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80317) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:08:12.692820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5577309ca9c0 pid: 80318 (default app) Jul 28 06:08:12.693490 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5577309ca9c0 pid: 80317 (default app) Jul 28 06:08:12.696852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16386d77-7928-45a6-826d-cd3c2b05fddb None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:12.697146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16386d77-7928-45a6-826d-cd3c2b05fddb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:12.697457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16386d77-7928-45a6-826d-cd3c2b05fddb None None] PATH_INFO: `/v3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:12.701751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1/1] 10.222.0.69 () {50 vars in 791 bytes} [Thu Jul 28 06:08:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:21.213378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8c301-e013-44b2-8229-41490a5e6297 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:21.213949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8c301-e013-44b2-8229-41490a5e6297 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:21.213949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8c301-e013-44b2-8229-41490a5e6297 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:21.218309 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1/2] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:21.223722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:21.224162 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:21.224294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:21.296068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: INFO dbcounter [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] Registered counter for database keystone Jul 28 06:08:21.318045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG oslo_db.sqlalchemy.engines [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80318) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 06:08:21.324562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writer thread running {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 06:08:21.506152 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG passlib.handlers.bcrypt [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80318) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 06:08:21.506585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG passlib.handlers.bcrypt [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80318) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 06:08:21.562720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:21.564358 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:21.693156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 2/3] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 06:08:21.833038 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8c301-e013-44b2-8229-41490a5e6297 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:21.903280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: INFO dbcounter [None req-36a8c301-e013-44b2-8229-41490a5e6297 None None] Registered counter for database keystone Jul 28 06:08:21.922517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG oslo_db.sqlalchemy.engines [None req-36a8c301-e013-44b2-8229-41490a5e6297 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80317) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 06:08:21.928798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writer thread running {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 06:08:22.048206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8c301-e013-44b2-8229-41490a5e6297 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:22.052550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11934d80-4947-47b8-aa6b-7a280021ecd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:22.053191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934d80-4947-47b8-aa6b-7a280021ecd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:22.053509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934d80-4947-47b8-aa6b-7a280021ecd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:22.053795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934d80-4947-47b8-aa6b-7a280021ecd3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:22.062134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11934d80-4947-47b8-aa6b-7a280021ecd3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:22.114359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WARNING keystone.common.rbac_enforcer.enforcer [None req-11934d80-4947-47b8-aa6b-7a280021ecd3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 06:08:22.114641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11934d80-4947-47b8-aa6b-7a280021ecd3 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:22.124862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 2/4] 10.222.0.69 () {58 vars in 1324 bytes} [Thu Jul 28 06:08:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 06:08:22.130869 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:22.146241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3ed0a-d43c-4255-9a8a-6a3988e5c4c6 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:22.151268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435825f2-f932-4723-bac1-7366d5d19270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:22.152386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435825f2-f932-4723-bac1-7366d5d19270 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:22.152678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435825f2-f932-4723-bac1-7366d5d19270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:22.152962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435825f2-f932-4723-bac1-7366d5d19270 admin admin] PATH_INFO: `/v3/projects/05bb534ab3954f32aad6d8b3751d3ac9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:22.154508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435825f2-f932-4723-bac1-7366d5d19270 admin admin] RBAC: Authorizing `identity:get_project(project_id=05bb534ab3954f32aad6d8b3751d3ac9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:22.205814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: WARNING keystone.common.rbac_enforcer.enforcer [None req-435825f2-f932-4723-bac1-7366d5d19270 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 06:08:22.206158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435825f2-f932-4723-bac1-7366d5d19270 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:22.211709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 3/5] 10.222.0.69 () {56 vars in 1207 bytes} [Thu Jul 28 06:08:22 2022] GET /identity/v3/projects/05bb534ab3954f32aad6d8b3751d3ac9 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:24.227613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549ac94-4d25-4f95-a0be-1213f7a92e27 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:24.228157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549ac94-4d25-4f95-a0be-1213f7a92e27 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:24.228157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549ac94-4d25-4f95-a0be-1213f7a92e27 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:24.232248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 3/6] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:08:24.237977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83adaef-d540-4e9c-b112-846f84bc1615 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:24.238416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83adaef-d540-4e9c-b112-846f84bc1615 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:24.238548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83adaef-d540-4e9c-b112-846f84bc1615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:24.259727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b83adaef-d540-4e9c-b112-846f84bc1615 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:24.261290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b83adaef-d540-4e9c-b112-846f84bc1615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:24.269645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 4/7] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 06:08:24.292308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549ac94-4d25-4f95-a0be-1213f7a92e27 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:24.304547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549ac94-4d25-4f95-a0be-1213f7a92e27 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:24.308686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ddc66-b68b-4ba0-8a20-c5370ad8263e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:24.309337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ddc66-b68b-4ba0-8a20-c5370ad8263e admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:24.309610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ddc66-b68b-4ba0-8a20-c5370ad8263e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:24.309889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ddc66-b68b-4ba0-8a20-c5370ad8263e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:24.317756 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ddc66-b68b-4ba0-8a20-c5370ad8263e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:24.319548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ddc66-b68b-4ba0-8a20-c5370ad8263e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:24.329067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 4/8] 10.222.0.69 () {58 vars in 1324 bytes} [Thu Jul 28 06:08:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:08:24.334827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83adaef-d540-4e9c-b112-846f84bc1615 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:24.344845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83adaef-d540-4e9c-b112-846f84bc1615 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:24.349541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354883e9-45e1-4dd1-890d-7349a5d9701c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:24.350202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354883e9-45e1-4dd1-890d-7349a5d9701c admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:24.350496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354883e9-45e1-4dd1-890d-7349a5d9701c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:24.350784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354883e9-45e1-4dd1-890d-7349a5d9701c admin admin] PATH_INFO: `/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:24.352936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354883e9-45e1-4dd1-890d-7349a5d9701c admin admin] RBAC: Authorizing `identity:get_user(user_id=b42a77304cdd4ceda1f777fdc9b5ccf9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:24.354831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354883e9-45e1-4dd1-890d-7349a5d9701c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:24.356429 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 5/9] 10.222.0.69 () {56 vars in 1198 bytes} [Thu Jul 28 06:08:24 2022] GET /identity/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:26.439766 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e3eb-90ca-4957-bcff-0f96bfd9f985 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:26.440298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e3eb-90ca-4957-bcff-0f96bfd9f985 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:26.440393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e3eb-90ca-4957-bcff-0f96bfd9f985 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:26.441708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 5/10] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:26.446828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43b64e-fe4b-47d5-b255-2e67eb382fe4 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:26.447465 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43b64e-fe4b-47d5-b255-2e67eb382fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:26.447754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43b64e-fe4b-47d5-b255-2e67eb382fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:26.448324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4455f8-52c7-42e5-8648-7d22ada9c05b None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:26.448618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4455f8-52c7-42e5-8648-7d22ada9c05b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:26.448903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4455f8-52c7-42e5-8648-7d22ada9c05b None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:26.450187 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 6/11] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:08:26.455201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba43b8-1c81-47d8-bd10-05e5a6ea12d7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:26.455496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba43b8-1c81-47d8-bd10-05e5a6ea12d7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:26.455787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba43b8-1c81-47d8-bd10-05e5a6ea12d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:26.467988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bc43b64e-fe4b-47d5-b255-2e67eb382fe4 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:26.469211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bc43b64e-fe4b-47d5-b255-2e67eb382fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:26.498386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG passlib.handlers.bcrypt [None req-8dba43b8-1c81-47d8-bd10-05e5a6ea12d7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80317) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 06:08:26.498844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG passlib.handlers.bcrypt [None req-8dba43b8-1c81-47d8-bd10-05e5a6ea12d7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80317) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 06:08:26.544954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-8dba43b8-1c81-47d8-bd10-05e5a6ea12d7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:26.546704 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8dba43b8-1c81-47d8-bd10-05e5a6ea12d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:26.550051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 6/12] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:08:26.573834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43b64e-fe4b-47d5-b255-2e67eb382fe4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:26.598441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:26.651080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 7/13] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:08:26.672330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba43b8-1c81-47d8-bd10-05e5a6ea12d7 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:26.682566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43b64e-fe4b-47d5-b255-2e67eb382fe4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:26.717534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d489a6-b577-421c-ba7a-b6cd690f1f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:26.718186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d489a6-b577-421c-ba7a-b6cd690f1f8c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:26.718470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d489a6-b577-421c-ba7a-b6cd690f1f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:26.718759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d489a6-b577-421c-ba7a-b6cd690f1f8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:26.730743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d489a6-b577-421c-ba7a-b6cd690f1f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-33d489a6-b577-421c-ba7a-b6cd690f1f8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:26.739598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:26.740514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:08:26.741696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 7/14] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:08:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:08:26.746721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d489a6-b577-421c-ba7a-b6cd690f1f8c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:26.775022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba43b8-1c81-47d8-bd10-05e5a6ea12d7 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:26.777350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:26.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9546921-c810-4d16-a9ca-4630b20c83bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:26.814958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9546921-c810-4d16-a9ca-4630b20c83bd None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:26.815250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9546921-c810-4d16-a9ca-4630b20c83bd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:26.815522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9546921-c810-4d16-a9ca-4630b20c83bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:26.831680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:26.850495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d489a6-b577-421c-ba7a-b6cd690f1f8c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:26.885640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b13b0-7aac-4da8-a7bd-c2d6af962163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:26.885875 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9546921-c810-4d16-a9ca-4630b20c83bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:26.886295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b13b0-7aac-4da8-a7bd-c2d6af962163 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:26.886588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b13b0-7aac-4da8-a7bd-c2d6af962163 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:26.886876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b13b0-7aac-4da8-a7bd-c2d6af962163 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:26.887276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9546921-c810-4d16-a9ca-4630b20c83bd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:26.887607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b13b0-7aac-4da8-a7bd-c2d6af962163 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:26.887922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b13b0-7aac-4da8-a7bd-c2d6af962163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:26.889344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b13b0-7aac-4da8-a7bd-c2d6af962163 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:26.902244 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 8/15] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:26.907710 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b13b0-7aac-4da8-a7bd-c2d6af962163 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:26.998741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 8/16] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:08:27.004304 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9546921-c810-4d16-a9ca-4630b20c83bd None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:27.007666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:27.015247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b13b0-7aac-4da8-a7bd-c2d6af962163 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:27.109186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9546921-c810-4d16-a9ca-4630b20c83bd None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:27.147563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339df749-88bc-460c-ade3-8442c0f48b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:27.148536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339df749-88bc-460c-ade3-8442c0f48b50 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:27.148822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339df749-88bc-460c-ade3-8442c0f48b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:27.149104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339df749-88bc-460c-ade3-8442c0f48b50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:27.160955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c3309-b787-47a4-a106-fd9b1a920bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:27.161522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c3309-b787-47a4-a106-fd9b1a920bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:27.161817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c3309-b787-47a4-a106-fd9b1a920bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:27.162095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c3309-b787-47a4-a106-fd9b1a920bc4 None admin] PATH_INFO: `/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:27.163308 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c3309-b787-47a4-a106-fd9b1a920bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=b42a77304cdd4ceda1f777fdc9b5ccf9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:27.164879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c3309-b787-47a4-a106-fd9b1a920bc4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:27.166604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 9/17] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:26 2022] GET /identity/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9 => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:27.167311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339df749-88bc-460c-ade3-8442c0f48b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:27.168782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339df749-88bc-460c-ade3-8442c0f48b50 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:27.172012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c3309-b787-47a4-a106-fd9b1a920bc4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:27.187215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-339df749-88bc-460c-ade3-8442c0f48b50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:08:27.188240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:27.189197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:27.190320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:08:27.190320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:08:27.190320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:08:27.190320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:27.190320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 9/18] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:08:27.195850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339df749-88bc-460c-ade3-8442c0f48b50 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:27.255813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:27.286958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c3309-b787-47a4-a106-fd9b1a920bc4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:27.310264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339df749-88bc-460c-ade3-8442c0f48b50 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:27.326935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2cb45-1cac-4d40-a547-3db9cda32fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:27.327601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2cb45-1cac-4d40-a547-3db9cda32fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:27.327892 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2cb45-1cac-4d40-a547-3db9cda32fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:27.328177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2cb45-1cac-4d40-a547-3db9cda32fb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:27.329498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2cb45-1cac-4d40-a547-3db9cda32fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:27.330944 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2cb45-1cac-4d40-a547-3db9cda32fb5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:27.332905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 10/19] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:27.349526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720aafc7-1996-4291-a7ec-a4e2cb929079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:27.350159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aafc7-1996-4291-a7ec-a4e2cb929079 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:27.350441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aafc7-1996-4291-a7ec-a4e2cb929079 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:27.351227 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aafc7-1996-4291-a7ec-a4e2cb929079 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:27.352108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720aafc7-1996-4291-a7ec-a4e2cb929079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:27.352434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720aafc7-1996-4291-a7ec-a4e2cb929079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:27.353203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2cb45-1cac-4d40-a547-3db9cda32fb5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:27.354135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720aafc7-1996-4291-a7ec-a4e2cb929079 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:27.376045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 10/20] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:27.447035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:27.467087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2cb45-1cac-4d40-a547-3db9cda32fb5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:27.503754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55011771-81e7-46b9-8d2f-826f95acf4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:27.504385 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55011771-81e7-46b9-8d2f-826f95acf4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:27.504668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55011771-81e7-46b9-8d2f-826f95acf4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:27.504952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55011771-81e7-46b9-8d2f-826f95acf4d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:27.505796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55011771-81e7-46b9-8d2f-826f95acf4d7 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.domain.id=default, user.id=b42a77304cdd4ceda1f777fdc9b5ccf9) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:27.506112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55011771-81e7-46b9-8d2f-826f95acf4d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:27.507655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55011771-81e7-46b9-8d2f-826f95acf4d7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:27.515334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 11/21] 10.222.0.69 () {56 vars in 1322 bytes} [Thu Jul 28 06:08:27 2022] GET /identity/v3/role_assignments?user.id=b42a77304cdd4ceda1f777fdc9b5ccf9&scope.domain.id=default&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:29.917413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988c51-c57a-4dd8-8cf7-6b03d66a4ad9 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:29.917962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988c51-c57a-4dd8-8cf7-6b03d66a4ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:29.917962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988c51-c57a-4dd8-8cf7-6b03d66a4ad9 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:29.919447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 11/22] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:29.924634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007078-1524-4ef9-8e24-2c0c8d377b11 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:29.925064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007078-1524-4ef9-8e24-2c0c8d377b11 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:29.925256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007078-1524-4ef9-8e24-2c0c8d377b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:29.950379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b9007078-1524-4ef9-8e24-2c0c8d377b11 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:29.951612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b9007078-1524-4ef9-8e24-2c0c8d377b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:30.037794 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 12/23] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:30.048255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee433-8986-4e59-8e6d-f931e9a8ff47 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:30.048558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee433-8986-4e59-8e6d-f931e9a8ff47 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:30.048849 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee433-8986-4e59-8e6d-f931e9a8ff47 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:30.050487 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 12/24] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:30.055906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d2a01-4220-4637-bfae-52e03401aaab None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:30.056211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d2a01-4220-4637-bfae-52e03401aaab None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:30.056503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d2a01-4220-4637-bfae-52e03401aaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:30.072691 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feee433-8986-4e59-8e6d-f931e9a8ff47 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:30.094369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-701d2a01-4220-4637-bfae-52e03401aaab None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:30.095748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-701d2a01-4220-4637-bfae-52e03401aaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:30.234677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feee433-8986-4e59-8e6d-f931e9a8ff47 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:30.257837 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:30.273280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686ec6a-858b-4122-9f65-95e0fd90f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:30.274392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ec6a-858b-4122-9f65-95e0fd90f675 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:30.274883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ec6a-858b-4122-9f65-95e0fd90f675 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:30.275378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ec6a-858b-4122-9f65-95e0fd90f675 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:30.276319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686ec6a-858b-4122-9f65-95e0fd90f675 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:30.278042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686ec6a-858b-4122-9f65-95e0fd90f675 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:30.291035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 13/25] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:30.292748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b1441-82c4-460b-81c3-efa6a27a7ae3 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:30.293266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b1441-82c4-460b-81c3-efa6a27a7ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:30.293769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b1441-82c4-460b-81c3-efa6a27a7ae3 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:30.295338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 14/26] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:30.314691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe96f3-e4d5-4104-868a-5857a5fc9884 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:30.315252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe96f3-e4d5-4104-868a-5857a5fc9884 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:30.321409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe96f3-e4d5-4104-868a-5857a5fc9884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:30.323824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:30.336738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6bfe96f3-e4d5-4104-868a-5857a5fc9884 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:30.338324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6bfe96f3-e4d5-4104-868a-5857a5fc9884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:30.447062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 15/27] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:08:30.447969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe96f3-e4d5-4104-868a-5857a5fc9884 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:30.581803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe96f3-e4d5-4104-868a-5857a5fc9884 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:30.618847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba509ae1-24fb-4922-83f1-cced8883d4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:30.619834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba509ae1-24fb-4922-83f1-cced8883d4aa None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:30.620314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba509ae1-24fb-4922-83f1-cced8883d4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:30.620784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba509ae1-24fb-4922-83f1-cced8883d4aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:30.622926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:30.625032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 13/28] 10.222.0.69 () {60 vars in 1131 bytes} [Thu Jul 28 06:08:30 2022] POST /identity/v3/roles => generated 219 bytes in 553 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:08:30.626587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b9327-eb71-4c47-ad11-8367d4fcc3b1 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:30.626885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b9327-eb71-4c47-ad11-8367d4fcc3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:30.627171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b9327-eb71-4c47-ad11-8367d4fcc3b1 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:30.628434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 14/29] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:30.629152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b9327-eb71-4c47-ad11-8367d4fcc3b1 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:30.636098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba509ae1-24fb-4922-83f1-cced8883d4aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:30.637970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba509ae1-24fb-4922-83f1-cced8883d4aa None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ba509ae1-24fb-4922-83f1-cced8883d4aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:08:30.658255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:30.705201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:30.706035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:08:30.706035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:08:30.706035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:08:30.706035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:08:30.706035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 16/30] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:30.706035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0af19-05e2-4a7c-8d88-dd555538697a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:30.706035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0af19-05e2-4a7c-8d88-dd555538697a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:30.706035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0af19-05e2-4a7c-8d88-dd555538697a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:30.706035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-59f0af19-05e2-4a7c-8d88-dd555538697a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:30.706509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-59f0af19-05e2-4a7c-8d88-dd555538697a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:30.815799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 17/31] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:30.816682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0af19-05e2-4a7c-8d88-dd555538697a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:30.856153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b9327-eb71-4c47-ad11-8367d4fcc3b1 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:30.906134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc110c5-7218-4d47-9b91-292bc62d6290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:30.907412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc110c5-7218-4d47-9b91-292bc62d6290 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:30.908236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc110c5-7218-4d47-9b91-292bc62d6290 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:30.908726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc110c5-7218-4d47-9b91-292bc62d6290 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:30.909961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc110c5-7218-4d47-9b91-292bc62d6290 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:30.911993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc110c5-7218-4d47-9b91-292bc62d6290 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:30.955561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0af19-05e2-4a7c-8d88-dd555538697a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:30.973936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:30.993725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b2fd6-1dd9-495c-a7ed-04e3bc879454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:30.994355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2fd6-1dd9-495c-a7ed-04e3bc879454 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:30.994698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2fd6-1dd9-495c-a7ed-04e3bc879454 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:30.994981 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2fd6-1dd9-495c-a7ed-04e3bc879454 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:30.995729 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b2fd6-1dd9-495c-a7ed-04e3bc879454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:30.996088 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b2fd6-1dd9-495c-a7ed-04e3bc879454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:30.997524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b2fd6-1dd9-495c-a7ed-04e3bc879454 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:31.003840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 15/32] 10.222.0.69 () {60 vars in 1131 bytes} [Thu Jul 28 06:08:30 2022] POST /identity/v3/roles => generated 225 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:08:31.005497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc110c5-7218-4d47-9b91-292bc62d6290 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:31.027806 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 18/33] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:31.034135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b2fd6-1dd9-495c-a7ed-04e3bc879454 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:31.056356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:31.146697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc110c5-7218-4d47-9b91-292bc62d6290 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:31.190686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7d3e5-31e9-4647-ba23-78cc463105a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:31.191781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7d3e5-31e9-4647-ba23-78cc463105a7 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.192330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7d3e5-31e9-4647-ba23-78cc463105a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.192818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7d3e5-31e9-4647-ba23-78cc463105a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.206829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7d3e5-31e9-4647-ba23-78cc463105a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:31.209810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b2fd6-1dd9-495c-a7ed-04e3bc879454 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-f7d7d3e5-31e9-4647-ba23-78cc463105a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:31.214695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:31.215701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:31.216456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:31.219200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 16/34] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:08:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:31.220785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c0ed-985e-4d2a-a64a-65e7452928d6 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.221185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c0ed-985e-4d2a-a64a-65e7452928d6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.221597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c0ed-985e-4d2a-a64a-65e7452928d6 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.223299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 17/35] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:31.225714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449c0ed-985e-4d2a-a64a-65e7452928d6 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:31.255234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46642e92-f2fa-4f4c-b2a5-41a987d84fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:31.255863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46642e92-f2fa-4f4c-b2a5-41a987d84fd0 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.256148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46642e92-f2fa-4f4c-b2a5-41a987d84fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.256428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46642e92-f2fa-4f4c-b2a5-41a987d84fd0 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.257932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46642e92-f2fa-4f4c-b2a5-41a987d84fd0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:31.259725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46642e92-f2fa-4f4c-b2a5-41a987d84fd0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:31.346755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:31.355260 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449c0ed-985e-4d2a-a64a-65e7452928d6 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:31.395730 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186af54-884b-4caa-8dc0-8e82e27a3682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:31.396761 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186af54-884b-4caa-8dc0-8e82e27a3682 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.397200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186af54-884b-4caa-8dc0-8e82e27a3682 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.397707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186af54-884b-4caa-8dc0-8e82e27a3682 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.398601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186af54-884b-4caa-8dc0-8e82e27a3682 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:31.399076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186af54-884b-4caa-8dc0-8e82e27a3682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:31.400822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186af54-884b-4caa-8dc0-8e82e27a3682 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:31.415276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 18/36] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:31.416827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3080722-3af4-4a6f-9d48-32365b637c45 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.417279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3080722-3af4-4a6f-9d48-32365b637c45 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.417715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3080722-3af4-4a6f-9d48-32365b637c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.438747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e3080722-3af4-4a6f-9d48-32365b637c45 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:31.440266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e3080722-3af4-4a6f-9d48-32365b637c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:31.492330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 19/37] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:08:31 2022] POST /identity/v3/projects => generated 301 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:08:31.493527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46642e92-f2fa-4f4c-b2a5-41a987d84fd0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:31.515028 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 19/38] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:31.515393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27628d-4e56-4995-8738-a09d9ac211b9 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.515883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27628d-4e56-4995-8738-a09d9ac211b9 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.516254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27628d-4e56-4995-8738-a09d9ac211b9 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.518058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 20/39] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:31.525896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aac864-e0e0-43bc-b2fa-daaa9622d231 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.526204 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aac864-e0e0-43bc-b2fa-daaa9622d231 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.526488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aac864-e0e0-43bc-b2fa-daaa9622d231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.543197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:31.546968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e9aac864-e0e0-43bc-b2fa-daaa9622d231 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:31.548301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e9aac864-e0e0-43bc-b2fa-daaa9622d231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:31.637495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 21/40] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:31.638707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aac864-e0e0-43bc-b2fa-daaa9622d231 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:31.639474 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46642e92-f2fa-4f4c-b2a5-41a987d84fd0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:31.664412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:31.684229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b29b79-c3cb-4007-8307-7969a3111d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:31.684896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b29b79-c3cb-4007-8307-7969a3111d46 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.685189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b29b79-c3cb-4007-8307-7969a3111d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.685501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b29b79-c3cb-4007-8307-7969a3111d46 None admin] PATH_INFO: `/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.686801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b29b79-c3cb-4007-8307-7969a3111d46 None admin] RBAC: Authorizing `identity:get_user(user_id=b42a77304cdd4ceda1f777fdc9b5ccf9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:31.688388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b29b79-c3cb-4007-8307-7969a3111d46 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:31.690125 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 20/41] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:31 2022] GET /identity/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:31.690969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b29b79-c3cb-4007-8307-7969a3111d46 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:31.752950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aac864-e0e0-43bc-b2fa-daaa9622d231 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:31.788163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bff3a-a418-426e-ad4e-20f2e5ca7e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:31.788984 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bff3a-a418-426e-ad4e-20f2e5ca7e68 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.789269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bff3a-a418-426e-ad4e-20f2e5ca7e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.789592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bff3a-a418-426e-ad4e-20f2e5ca7e68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.791034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28bff3a-a418-426e-ad4e-20f2e5ca7e68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:31.791736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:31.792655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28bff3a-a418-426e-ad4e-20f2e5ca7e68 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:31.794174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 22/42] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:31.794942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bff3a-a418-426e-ad4e-20f2e5ca7e68 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:31.798588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b29b79-c3cb-4007-8307-7969a3111d46 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:31.836197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37199c-a03d-4ffb-9646-30a568e05818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:31.836834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37199c-a03d-4ffb-9646-30a568e05818 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.837127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37199c-a03d-4ffb-9646-30a568e05818 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.837467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37199c-a03d-4ffb-9646-30a568e05818 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.838198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37199c-a03d-4ffb-9646-30a568e05818 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:31.839613 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37199c-a03d-4ffb-9646-30a568e05818 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:31.884064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:31.899027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bff3a-a418-426e-ad4e-20f2e5ca7e68 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:31.912812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 21/43] 10.222.0.69 () {60 vars in 1131 bytes} [Thu Jul 28 06:08:31 2022] POST /identity/v3/roles => generated 223 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:08:31.913719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37199c-a03d-4ffb-9646-30a568e05818 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:31.937823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a7489-ebd5-4656-a384-38875d1acd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:31.937823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a7489-ebd5-4656-a384-38875d1acd8e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:31.937823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a7489-ebd5-4656-a384-38875d1acd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:31.938191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a7489-ebd5-4656-a384-38875d1acd8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:31.939550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a7489-ebd5-4656-a384-38875d1acd8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:31.941093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a7489-ebd5-4656-a384-38875d1acd8e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:31.942848 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 23/44] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:31.949575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a7489-ebd5-4656-a384-38875d1acd8e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:32.038556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37199c-a03d-4ffb-9646-30a568e05818 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:32.067355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a7489-ebd5-4656-a384-38875d1acd8e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:32.081404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63593f7d-40ac-4243-9199-58c85588572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:32.082396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63593f7d-40ac-4243-9199-58c85588572e None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:32.082396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63593f7d-40ac-4243-9199-58c85588572e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:32.082599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63593f7d-40ac-4243-9199-58c85588572e None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:32.083305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63593f7d-40ac-4243-9199-58c85588572e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:32.084783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63593f7d-40ac-4243-9199-58c85588572e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:32.107464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956003db-773b-48c2-a404-e460331fc66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:32.107664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956003db-773b-48c2-a404-e460331fc66f None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:32.107913 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956003db-773b-48c2-a404-e460331fc66f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:32.108234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956003db-773b-48c2-a404-e460331fc66f None admin] PATH_INFO: `/v3/domains/default/users/b42a77304cdd4ceda1f777fdc9b5ccf9/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:32.110636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956003db-773b-48c2-a404-e460331fc66f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b42a77304cdd4ceda1f777fdc9b5ccf9, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:32.112154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956003db-773b-48c2-a404-e460331fc66f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:32.199667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 22/45] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:08:31 2022] POST /identity/v3/projects => generated 312 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:08:32.210301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 24/46] 10.222.0.69 () {58 vars in 1361 bytes} [Thu Jul 28 06:08:31 2022] PUT /identity/v3/domains/default/users/b42a77304cdd4ceda1f777fdc9b5ccf9/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:08:34.238426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f5727-fd4c-41e9-9fb7-9707c9b7122c None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:34.238946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f5727-fd4c-41e9-9fb7-9707c9b7122c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:34.239053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f5727-fd4c-41e9-9fb7-9707c9b7122c None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:34.240364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 23/47] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:34.245873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97617ab0-04d2-4aeb-ada3-b6670e091f1b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:34.246302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97617ab0-04d2-4aeb-ada3-b6670e091f1b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:34.246447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97617ab0-04d2-4aeb-ada3-b6670e091f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:34.256817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d036b1-8f96-421d-b9f5-ee75bc65c721 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:34.257111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d036b1-8f96-421d-b9f5-ee75bc65c721 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:34.257432 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d036b1-8f96-421d-b9f5-ee75bc65c721 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:34.258782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 24/48] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:08:34.263847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4005d5-1132-4cbb-8e64-d8e445ae5c02 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:34.264137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4005d5-1132-4cbb-8e64-d8e445ae5c02 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:34.264423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4005d5-1132-4cbb-8e64-d8e445ae5c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:34.265494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-97617ab0-04d2-4aeb-ada3-b6670e091f1b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:34.266952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-97617ab0-04d2-4aeb-ada3-b6670e091f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:34.278811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:34.285098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6f4005d5-1132-4cbb-8e64-d8e445ae5c02 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:34.286656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6f4005d5-1132-4cbb-8e64-d8e445ae5c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:34.289474 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:34.338374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 25/49] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:34.355346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 25/50] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:08:34.362950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97617ab0-04d2-4aeb-ada3-b6670e091f1b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:34.378324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4005d5-1132-4cbb-8e64-d8e445ae5c02 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:34.465915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97617ab0-04d2-4aeb-ada3-b6670e091f1b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:34.482510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4005d5-1132-4cbb-8e64-d8e445ae5c02 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:34.502408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7dd07-10c9-4233-9fae-b1ae78be06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:34.503078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7dd07-10c9-4233-9fae-b1ae78be06a4 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:34.507237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7dd07-10c9-4233-9fae-b1ae78be06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:34.507237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7dd07-10c9-4233-9fae-b1ae78be06a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:34.507237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7dd07-10c9-4233-9fae-b1ae78be06a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:34.509324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7dd07-10c9-4233-9fae-b1ae78be06a4 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:34.511001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 26/51] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:34.517030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7dd07-10c9-4233-9fae-b1ae78be06a4 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:34.525875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997b712-771b-4cbb-8f60-438e03f5d906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:34.526549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997b712-771b-4cbb-8f60-438e03f5d906 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:34.526837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997b712-771b-4cbb-8f60-438e03f5d906 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:34.527121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997b712-771b-4cbb-8f60-438e03f5d906 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:34.532997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:34.534425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:34.538128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997b712-771b-4cbb-8f60-438e03f5d906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6997b712-771b-4cbb-8f60-438e03f5d906 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:34.543061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:34.550828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 26/52] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:08:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:08:34.551770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997b712-771b-4cbb-8f60-438e03f5d906 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:34.622854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7dd07-10c9-4233-9fae-b1ae78be06a4 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:34.652452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997b712-771b-4cbb-8f60-438e03f5d906 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:34.658166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a63ea0-0230-4d53-8635-9bed13fb03ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:34.658878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a63ea0-0230-4d53-8635-9bed13fb03ab None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:34.659237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a63ea0-0230-4d53-8635-9bed13fb03ab None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:34.659530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a63ea0-0230-4d53-8635-9bed13fb03ab None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:34.660670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a63ea0-0230-4d53-8635-9bed13fb03ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:34.662170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a63ea0-0230-4d53-8635-9bed13fb03ab None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:34.686753 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e5f36-3598-4791-a7ab-0f3d79a11490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:34.687394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e5f36-3598-4791-a7ab-0f3d79a11490 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:34.687681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e5f36-3598-4791-a7ab-0f3d79a11490 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:34.687967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e5f36-3598-4791-a7ab-0f3d79a11490 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:34.688698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e5f36-3598-4791-a7ab-0f3d79a11490 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:34.689009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e5f36-3598-4791-a7ab-0f3d79a11490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:34.690461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e5f36-3598-4791-a7ab-0f3d79a11490 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:34.699447 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 27/53] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:34.705135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e5f36-3598-4791-a7ab-0f3d79a11490 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:34.750141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 27/54] 10.222.0.69 () {60 vars in 1140 bytes} [Thu Jul 28 06:08:34 2022] POST /identity/v3/projects => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:08:34.767850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:34.813990 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e5f36-3598-4791-a7ab-0f3d79a11490 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:34.861778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52167b7d-a376-4d79-837c-767511208f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:34.862453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52167b7d-a376-4d79-837c-767511208f88 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:34.862743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52167b7d-a376-4d79-837c-767511208f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:34.863026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52167b7d-a376-4d79-837c-767511208f88 None admin] PATH_INFO: `/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:34.864288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52167b7d-a376-4d79-837c-767511208f88 None admin] RBAC: Authorizing `identity:get_user(user_id=b42a77304cdd4ceda1f777fdc9b5ccf9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:34.866459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52167b7d-a376-4d79-837c-767511208f88 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:34.868224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 28/55] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:34 2022] GET /identity/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:34.874024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a63ea0-0230-4d53-8635-9bed13fb03ab None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:34.943180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:34.981876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a63ea0-0230-4d53-8635-9bed13fb03ab None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:35.016452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb10b35-030b-4fa6-9733-41a5dfbd925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:35.017116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb10b35-030b-4fa6-9733-41a5dfbd925e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:35.017458 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb10b35-030b-4fa6-9733-41a5dfbd925e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:35.017736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb10b35-030b-4fa6-9733-41a5dfbd925e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:35.018989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb10b35-030b-4fa6-9733-41a5dfbd925e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:35.020420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb10b35-030b-4fa6-9733-41a5dfbd925e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:35.022054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 28/56] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:35.027767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52167b7d-a376-4d79-837c-767511208f88 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:35.129004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52167b7d-a376-4d79-837c-767511208f88 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:35.155960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:35.164457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2399c2-f040-480d-849c-ac13612c3a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:35.165088 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2399c2-f040-480d-849c-ac13612c3a31 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:35.165412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2399c2-f040-480d-849c-ac13612c3a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:35.165693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2399c2-f040-480d-849c-ac13612c3a31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:35.166498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2399c2-f040-480d-849c-ac13612c3a31 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.domain.id=default, user.id=b42a77304cdd4ceda1f777fdc9b5ccf9) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:35.171953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2399c2-f040-480d-849c-ac13612c3a31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:35.171953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2399c2-f040-480d-849c-ac13612c3a31 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:35.185344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 29/57] 10.222.0.69 () {56 vars in 1322 bytes} [Thu Jul 28 06:08:35 2022] GET /identity/v3/role_assignments?user.id=b42a77304cdd4ceda1f777fdc9b5ccf9&scope.domain.id=default&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:36.789041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804b1c7-df83-4b83-ba5f-c742c72535d1 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:36.819685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804b1c7-df83-4b83-ba5f-c742c72535d1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:36.819685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804b1c7-df83-4b83-ba5f-c742c72535d1 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:36.819685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 29/58] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:36.819965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7717e-8abe-4e5b-a7aa-5f2a23df9d5b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:36.819965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7717e-8abe-4e5b-a7aa-5f2a23df9d5b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:36.819965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7717e-8abe-4e5b-a7aa-5f2a23df9d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:36.819965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-07e7717e-8abe-4e5b-a7aa-5f2a23df9d5b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:36.819965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-07e7717e-8abe-4e5b-a7aa-5f2a23df9d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:36.882846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 30/59] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:36.906880 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804b1c7-df83-4b83-ba5f-c742c72535d1 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:37.007932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804b1c7-df83-4b83-ba5f-c742c72535d1 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:37.036588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:37.041686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f280523-0863-4a61-b820-b94329ab9a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:37.042282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280523-0863-4a61-b820-b94329ab9a6f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:37.042575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280523-0863-4a61-b820-b94329ab9a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:37.042862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280523-0863-4a61-b820-b94329ab9a6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:37.044031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f280523-0863-4a61-b820-b94329ab9a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:37.045517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f280523-0863-4a61-b820-b94329ab9a6f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:37.047131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 30/60] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:37.052717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7717e-8abe-4e5b-a7aa-5f2a23df9d5b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:37.150966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7717e-8abe-4e5b-a7aa-5f2a23df9d5b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:37.172776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:37.184164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747f6b6-f4e2-4e9b-b478-f4c2d8b54fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:37.186374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747f6b6-f4e2-4e9b-b478-f4c2d8b54fab None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:37.186374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747f6b6-f4e2-4e9b-b478-f4c2d8b54fab None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:37.186374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747f6b6-f4e2-4e9b-b478-f4c2d8b54fab None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:37.186374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747f6b6-f4e2-4e9b-b478-f4c2d8b54fab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:37.187477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747f6b6-f4e2-4e9b-b478-f4c2d8b54fab None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:37.285660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 31/61] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:08:37 2022] POST /identity/v3/users => generated 275 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:08:40.471375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0111e2-d3f5-43a0-909c-76ab62a16d66 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:40.472272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0111e2-d3f5-43a0-909c-76ab62a16d66 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:40.472723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0111e2-d3f5-43a0-909c-76ab62a16d66 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:40.474849 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 31/62] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:40.480499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb1650-0436-41dc-8e2a-410b538c8e1d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:40.481430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb1650-0436-41dc-8e2a-410b538c8e1d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:40.481934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb1650-0436-41dc-8e2a-410b538c8e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:40.506222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5dfb1650-0436-41dc-8e2a-410b538c8e1d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:40.508047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5dfb1650-0436-41dc-8e2a-410b538c8e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:40.599744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 32/63] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:40.635256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0111e2-d3f5-43a0-909c-76ab62a16d66 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:40.667420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f972686-c276-4062-bd8f-2476c9405a5c None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:40.668075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f972686-c276-4062-bd8f-2476c9405a5c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:40.668576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f972686-c276-4062-bd8f-2476c9405a5c None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:40.670345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 33/64] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:40.677754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f75c24-d21c-4094-87d0-800e7e7eeba6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:40.678945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f75c24-d21c-4094-87d0-800e7e7eeba6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:40.679436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f75c24-d21c-4094-87d0-800e7e7eeba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:40.699101 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d8f75c24-d21c-4094-87d0-800e7e7eeba6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:40.700659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d8f75c24-d21c-4094-87d0-800e7e7eeba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:40.786235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 34/65] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:40.789385 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f2dae-2509-407c-9b09-cf62480dfe04 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:40.790044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f2dae-2509-407c-9b09-cf62480dfe04 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:40.790513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f2dae-2509-407c-9b09-cf62480dfe04 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:40.791275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0111e2-d3f5-43a0-909c-76ab62a16d66 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:40.796250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 35/66] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:40.798734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef388a0-64dd-4e53-9b0e-5919a9cfd3b3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:40.799253 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef388a0-64dd-4e53-9b0e-5919a9cfd3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:40.799769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef388a0-64dd-4e53-9b0e-5919a9cfd3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:40.826471 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7ef388a0-64dd-4e53-9b0e-5919a9cfd3b3 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:40.827689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7ef388a0-64dd-4e53-9b0e-5919a9cfd3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:40.839940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed59b3-a5d3-48aa-9e1d-2ac48403d4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:40.840376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed59b3-a5d3-48aa-9e1d-2ac48403d4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:40.840665 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed59b3-a5d3-48aa-9e1d-2ac48403d4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:40.840938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed59b3-a5d3-48aa-9e1d-2ac48403d4d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:40.853833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed59b3-a5d3-48aa-9e1d-2ac48403d4d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-e4ed59b3-a5d3-48aa-9e1d-2ac48403d4d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:40.859226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:40.860111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:40.860903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 32/67] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:08:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:40.861590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed59b3-a5d3-48aa-9e1d-2ac48403d4d6 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:40.867826 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:40.910287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 36/68] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:40.912189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b65136-9d18-4c84-ac5d-57ebb015f14a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:40.912494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b65136-9d18-4c84-ac5d-57ebb015f14a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:40.912996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b65136-9d18-4c84-ac5d-57ebb015f14a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:40.914306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 37/69] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:40.915161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b65136-9d18-4c84-ac5d-57ebb015f14a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:40.936373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:40.980667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed59b3-a5d3-48aa-9e1d-2ac48403d4d6 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:41.015852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba994c42-8fb3-4cc7-ba36-eec699a230ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:41.016638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba994c42-8fb3-4cc7-ba36-eec699a230ed None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.016900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba994c42-8fb3-4cc7-ba36-eec699a230ed None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.017105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba994c42-8fb3-4cc7-ba36-eec699a230ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.026168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b65136-9d18-4c84-ac5d-57ebb015f14a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:41.028501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba994c42-8fb3-4cc7-ba36-eec699a230ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-ba994c42-8fb3-4cc7-ba36-eec699a230ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:41.033927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:41.034796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 33/70] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:08:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:41.041099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0c8dc-7010-4562-9577-f6d8d498aea9 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.041851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0c8dc-7010-4562-9577-f6d8d498aea9 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.041851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0c8dc-7010-4562-9577-f6d8d498aea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.055474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-cdd0c8dc-7010-4562-9577-f6d8d498aea9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:41.056684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-cdd0c8dc-7010-4562-9577-f6d8d498aea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:41.072359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae5150-d1fd-4305-8231-d28301c7068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:41.073023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae5150-d1fd-4305-8231-d28301c7068d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.073287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae5150-d1fd-4305-8231-d28301c7068d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.073598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae5150-d1fd-4305-8231-d28301c7068d None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.074328 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae5150-d1fd-4305-8231-d28301c7068d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:41.074640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae5150-d1fd-4305-8231-d28301c7068d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:41.076052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae5150-d1fd-4305-8231-d28301c7068d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:41.084771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 38/71] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:08:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:41.085562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae5150-d1fd-4305-8231-d28301c7068d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:41.118426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:41.137886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 34/72] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:41.139043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0c8dc-7010-4562-9577-f6d8d498aea9 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:41.209069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:41.215511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae5150-d1fd-4305-8231-d28301c7068d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:41.250767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca38931-a161-4901-9599-89296731fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:41.251515 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca38931-a161-4901-9599-89296731fb70 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.251806 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca38931-a161-4901-9599-89296731fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.252089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca38931-a161-4901-9599-89296731fb70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.260167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0c8dc-7010-4562-9577-f6d8d498aea9 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:41.263155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca38931-a161-4901-9599-89296731fb70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-9ca38931-a161-4901-9599-89296731fb70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:41.268795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:41.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:08:41.270472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 39/73] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:41.271095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca38931-a161-4901-9599-89296731fb70 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:41.297454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491888b-3cdb-4452-bd05-3efbd6713615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:41.298095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491888b-3cdb-4452-bd05-3efbd6713615 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.298391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491888b-3cdb-4452-bd05-3efbd6713615 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.298673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491888b-3cdb-4452-bd05-3efbd6713615 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.299411 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491888b-3cdb-4452-bd05-3efbd6713615 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:41.299728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491888b-3cdb-4452-bd05-3efbd6713615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:41.301196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491888b-3cdb-4452-bd05-3efbd6713615 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:41.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 35/74] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:41.310947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491888b-3cdb-4452-bd05-3efbd6713615 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:41.364954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:41.375006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca38931-a161-4901-9599-89296731fb70 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:41.410456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2966f02-afa3-42c9-89d6-3337de42a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:41.411111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2966f02-afa3-42c9-89d6-3337de42a428 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.411400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2966f02-afa3-42c9-89d6-3337de42a428 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.411690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2966f02-afa3-42c9-89d6-3337de42a428 None admin] PATH_INFO: `/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.421245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491888b-3cdb-4452-bd05-3efbd6713615 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:41.430535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2966f02-afa3-42c9-89d6-3337de42a428 None admin] RBAC: Authorizing `identity:get_user(user_id=05a7b57c5cfe4d368e6c200fc07ecf2d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:41.432051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2966f02-afa3-42c9-89d6-3337de42a428 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:41.438600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 40/75] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:41 2022] GET /identity/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:41.438600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c310b-345d-41b1-a1ad-bd3b6ce46c7f None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.438600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c310b-345d-41b1-a1ad-bd3b6ce46c7f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.438600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c310b-345d-41b1-a1ad-bd3b6ce46c7f None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.438600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 41/76] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:41.438600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c310b-345d-41b1-a1ad-bd3b6ce46c7f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:41.442234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:41.466947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760ebe9-5000-4e02-bb07-37d1a05d1373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:41.467524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760ebe9-5000-4e02-bb07-37d1a05d1373 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.467815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760ebe9-5000-4e02-bb07-37d1a05d1373 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.468102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760ebe9-5000-4e02-bb07-37d1a05d1373 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.479657 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760ebe9-5000-4e02-bb07-37d1a05d1373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-b760ebe9-5000-4e02-bb07-37d1a05d1373 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:41.485207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:41.486082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:41.486845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 36/77] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:41.487465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760ebe9-5000-4e02-bb07-37d1a05d1373 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:41.548849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c310b-345d-41b1-a1ad-bd3b6ce46c7f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:41.582913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adceb74b-f7c6-4fed-a922-c14b3bbe823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:41.583538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adceb74b-f7c6-4fed-a922-c14b3bbe823f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.583833 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adceb74b-f7c6-4fed-a922-c14b3bbe823f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.584120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adceb74b-f7c6-4fed-a922-c14b3bbe823f None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.585426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adceb74b-f7c6-4fed-a922-c14b3bbe823f None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:41.585851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adceb74b-f7c6-4fed-a922-c14b3bbe823f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:41.587514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adceb74b-f7c6-4fed-a922-c14b3bbe823f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:41.595559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 42/78] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:08:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:41.595777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760ebe9-5000-4e02-bb07-37d1a05d1373 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:41.596374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adceb74b-f7c6-4fed-a922-c14b3bbe823f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:41.609156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:41.633017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499095ee-d237-4f25-9ace-2903011bf05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:41.633810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499095ee-d237-4f25-9ace-2903011bf05f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.634101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499095ee-d237-4f25-9ace-2903011bf05f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.634387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499095ee-d237-4f25-9ace-2903011bf05f None admin] PATH_INFO: `/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.635589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499095ee-d237-4f25-9ace-2903011bf05f None admin] RBAC: Authorizing `identity:get_user(user_id=b42a77304cdd4ceda1f777fdc9b5ccf9)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:41.637106 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499095ee-d237-4f25-9ace-2903011bf05f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:41.638914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 37/79] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:41 2022] GET /identity/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:41.639883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e194f-e6db-4645-885c-b1a91d141e99 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.640175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e194f-e6db-4645-885c-b1a91d141e99 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.640456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e194f-e6db-4645-885c-b1a91d141e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.660471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e01e194f-e6db-4645-885c-b1a91d141e99 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:41.661722 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e01e194f-e6db-4645-885c-b1a91d141e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:41.696668 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:41.701752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adceb74b-f7c6-4fed-a922-c14b3bbe823f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:41.729749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 38/80] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:41.730520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e194f-e6db-4645-885c-b1a91d141e99 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:41.741099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dfaec-5ca9-4f26-87e4-f1b15478c650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:41.741802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dfaec-5ca9-4f26-87e4-f1b15478c650 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.742079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dfaec-5ca9-4f26-87e4-f1b15478c650 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.742358 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dfaec-5ca9-4f26-87e4-f1b15478c650 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.811168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dfaec-5ca9-4f26-87e4-f1b15478c650 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:41.812666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dfaec-5ca9-4f26-87e4-f1b15478c650 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:41.825294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:41.847431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e194f-e6db-4645-885c-b1a91d141e99 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:41.883406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d527c8-9faa-4e73-8b9d-9df33f1e9214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:41.884036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d527c8-9faa-4e73-8b9d-9df33f1e9214 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:41.884323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d527c8-9faa-4e73-8b9d-9df33f1e9214 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:41.884599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d527c8-9faa-4e73-8b9d-9df33f1e9214 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:41.885335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d527c8-9faa-4e73-8b9d-9df33f1e9214 None admin] RBAC: Adding query filter params (name=member) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:41.885655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d527c8-9faa-4e73-8b9d-9df33f1e9214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:41.887031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d527c8-9faa-4e73-8b9d-9df33f1e9214 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:41.895487 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 39/81] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:08:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:41.896285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d527c8-9faa-4e73-8b9d-9df33f1e9214 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:41.921159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 43/82] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:41.922051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dfaec-5ca9-4f26-87e4-f1b15478c650 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:41.926894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:41.995445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d527c8-9faa-4e73-8b9d-9df33f1e9214 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:42.022277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dfaec-5ca9-4f26-87e4-f1b15478c650 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:42.029261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ca245-813f-4eaf-99ad-a2c9a9109b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:42.029940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ca245-813f-4eaf-99ad-a2c9a9109b50 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:42.030237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ca245-813f-4eaf-99ad-a2c9a9109b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:42.030521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ca245-813f-4eaf-99ad-a2c9a9109b50 None admin] PATH_INFO: `/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:42.031709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ca245-813f-4eaf-99ad-a2c9a9109b50 None admin] RBAC: Authorizing `identity:get_user(user_id=05a7b57c5cfe4d368e6c200fc07ecf2d)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:42.033174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ca245-813f-4eaf-99ad-a2c9a9109b50 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:42.033297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:42.034824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 40/83] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:41 2022] GET /identity/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:42.035597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ca245-813f-4eaf-99ad-a2c9a9109b50 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:42.061538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88568b75-6bdb-4e5b-ba28-e42bde8efdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:42.062170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88568b75-6bdb-4e5b-ba28-e42bde8efdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:42.062458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88568b75-6bdb-4e5b-ba28-e42bde8efdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:42.062740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88568b75-6bdb-4e5b-ba28-e42bde8efdc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:42.134369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88568b75-6bdb-4e5b-ba28-e42bde8efdc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:42.136126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88568b75-6bdb-4e5b-ba28-e42bde8efdc1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:42.137582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:42.142460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ca245-813f-4eaf-99ad-a2c9a9109b50 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:42.178081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4a20e-74e7-4bad-b36a-7320981f2b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:42.179381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4a20e-74e7-4bad-b36a-7320981f2b50 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:42.179674 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4a20e-74e7-4bad-b36a-7320981f2b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:42.179962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4a20e-74e7-4bad-b36a-7320981f2b50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:42.181158 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4a20e-74e7-4bad-b36a-7320981f2b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:42.182683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4a20e-74e7-4bad-b36a-7320981f2b50 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:42.188881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 41/84] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:42.188881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4a20e-74e7-4bad-b36a-7320981f2b50 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:42.260231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 44/85] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:42.261096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88568b75-6bdb-4e5b-ba28-e42bde8efdc1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:42.273733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:42.303432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4a20e-74e7-4bad-b36a-7320981f2b50 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:42.339184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14909348-e6c3-4a51-b890-054d89dd07e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:42.339853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14909348-e6c3-4a51-b890-054d89dd07e3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:42.340157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14909348-e6c3-4a51-b890-054d89dd07e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:42.340442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14909348-e6c3-4a51-b890-054d89dd07e3 None admin] PATH_INFO: `/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:42.341685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14909348-e6c3-4a51-b890-054d89dd07e3 None admin] RBAC: Authorizing `identity:get_user(user_id=05a7b57c5cfe4d368e6c200fc07ecf2d)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:42.343166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14909348-e6c3-4a51-b890-054d89dd07e3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:42.344978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 42/86] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:42 2022] GET /identity/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:42.345596 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14909348-e6c3-4a51-b890-054d89dd07e3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:42.374249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88568b75-6bdb-4e5b-ba28-e42bde8efdc1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:42.384515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:42.416099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e30ab-cdcc-4e9e-9855-23b22c6668cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:42.416728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e30ab-cdcc-4e9e-9855-23b22c6668cb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:42.417021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e30ab-cdcc-4e9e-9855-23b22c6668cb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:42.417324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e30ab-cdcc-4e9e-9855-23b22c6668cb None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:42.419617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e30ab-cdcc-4e9e-9855-23b22c6668cb None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:42.421120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e30ab-cdcc-4e9e-9855-23b22c6668cb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:42.423055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 45/87] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:42 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:42.428774 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e30ab-cdcc-4e9e-9855-23b22c6668cb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:42.459909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14909348-e6c3-4a51-b890-054d89dd07e3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:42.501255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58606be1-a655-4963-b8f7-541dcb34e678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:42.501925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58606be1-a655-4963-b8f7-541dcb34e678 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:42.502184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58606be1-a655-4963-b8f7-541dcb34e678 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:42.502462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58606be1-a655-4963-b8f7-541dcb34e678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:42.528803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:42.550829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e30ab-cdcc-4e9e-9855-23b22c6668cb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:42.576256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58606be1-a655-4963-b8f7-541dcb34e678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:42.577772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58606be1-a655-4963-b8f7-541dcb34e678 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:42.586829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9a88a-85de-4cf2-ae97-2a94c93f653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:42.587458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9a88a-85de-4cf2-ae97-2a94c93f653e None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:42.587747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9a88a-85de-4cf2-ae97-2a94c93f653e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:42.588027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9a88a-85de-4cf2-ae97-2a94c93f653e None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:42.588721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9a88a-85de-4cf2-ae97-2a94c93f653e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:42.590212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9a88a-85de-4cf2-ae97-2a94c93f653e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:42.610758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:42.674349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 43/88] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:42.679593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58606be1-a655-4963-b8f7-541dcb34e678 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:42.679682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 46/89] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:08:42 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:08:42.679682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9a88a-85de-4cf2-ae97-2a94c93f653e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:42.795014 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58606be1-a655-4963-b8f7-541dcb34e678 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:42.802019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9a88a-85de-4cf2-ae97-2a94c93f653e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:42.838156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ae328-fbc4-4328-8170-190138e67561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:42.838821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ae328-fbc4-4328-8170-190138e67561 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:42.839182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ae328-fbc4-4328-8170-190138e67561 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:42.839465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ae328-fbc4-4328-8170-190138e67561 None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:42.840690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ae328-fbc4-4328-8170-190138e67561 None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:42.842284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ae328-fbc4-4328-8170-190138e67561 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:42.844060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 44/90] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:42 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:42.844867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ae328-fbc4-4328-8170-190138e67561 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:42.852623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a7b4c-9317-4f12-a32d-b89ff9bd1e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:42.853268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a7b4c-9317-4f12-a32d-b89ff9bd1e22 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:42.853615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a7b4c-9317-4f12-a32d-b89ff9bd1e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:42.853999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a7b4c-9317-4f12-a32d-b89ff9bd1e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:42.858071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:42.862267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:42.944997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a7b4c-9317-4f12-a32d-b89ff9bd1e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:42.946547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a7b4c-9317-4f12-a32d-b89ff9bd1e22 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:42.979055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ae328-fbc4-4328-8170-190138e67561 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:43.019729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b81388-cb06-4760-b9f2-998e5e11f718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:43.020364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b81388-cb06-4760-b9f2-998e5e11f718 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:43.020659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b81388-cb06-4760-b9f2-998e5e11f718 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:43.020941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b81388-cb06-4760-b9f2-998e5e11f718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:43.022486 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b81388-cb06-4760-b9f2-998e5e11f718 None admin] RBAC: Adding query filter params (role.id=c7b4bfc82f3b46cfaa10773dad7cd03a, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3, user.id=05a7b57c5cfe4d368e6c200fc07ecf2d) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:43.022812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b81388-cb06-4760-b9f2-998e5e11f718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:43.024283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b81388-cb06-4760-b9f2-998e5e11f718 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:43.032307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 45/91] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:42 2022] GET /identity/v3/role_assignments?user.id=05a7b57c5cfe4d368e6c200fc07ecf2d&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=c7b4bfc82f3b46cfaa10773dad7cd03a => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:43.033207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b81388-cb06-4760-b9f2-998e5e11f718 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:43.083813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 47/92] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:43.085769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a7b4c-9317-4f12-a32d-b89ff9bd1e22 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:43.111387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:43.128228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:43.175688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b81388-cb06-4760-b9f2-998e5e11f718 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:43.210267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a7b4c-9317-4f12-a32d-b89ff9bd1e22 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:43.249789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ac425-a344-46b6-9dc2-3bb8703252e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:43.250010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c270a2-99af-40ed-8ea4-cc2186cf953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:43.250646 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c270a2-99af-40ed-8ea4-cc2186cf953a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:43.250941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c270a2-99af-40ed-8ea4-cc2186cf953a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:43.251143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ac425-a344-46b6-9dc2-3bb8703252e5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:43.251254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c270a2-99af-40ed-8ea4-cc2186cf953a None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:43.251655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ac425-a344-46b6-9dc2-3bb8703252e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:43.252079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ac425-a344-46b6-9dc2-3bb8703252e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:43.252429 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c270a2-99af-40ed-8ea4-cc2186cf953a None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:43.252888 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ac425-a344-46b6-9dc2-3bb8703252e5 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3, user.id=b42a77304cdd4ceda1f777fdc9b5ccf9) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:43.253203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ac425-a344-46b6-9dc2-3bb8703252e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:43.253954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c270a2-99af-40ed-8ea4-cc2186cf953a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:43.254674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ac425-a344-46b6-9dc2-3bb8703252e5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:43.261195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 46/93] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:43 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:43.261730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 48/94] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:43 2022] GET /identity/v3/role_assignments?user.id=b42a77304cdd4ceda1f777fdc9b5ccf9&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:43.262123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c270a2-99af-40ed-8ea4-cc2186cf953a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:43.262539 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ac425-a344-46b6-9dc2-3bb8703252e5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:43.388054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:43.398965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ac425-a344-46b6-9dc2-3bb8703252e5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:43.430642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c270a2-99af-40ed-8ea4-cc2186cf953a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:43.466166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1834f-6336-40aa-b335-aa97dd834095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:43.466830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1834f-6336-40aa-b335-aa97dd834095 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:43.467138 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1834f-6336-40aa-b335-aa97dd834095 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:43.467414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1834f-6336-40aa-b335-aa97dd834095 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:43.468236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1834f-6336-40aa-b335-aa97dd834095 None admin] RBAC: Adding query filter params (role.id=c4fc73849d664b2588bd2de7372aa839, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3, user.id=05a7b57c5cfe4d368e6c200fc07ecf2d) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:43.468562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1834f-6336-40aa-b335-aa97dd834095 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:43.470171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1834f-6336-40aa-b335-aa97dd834095 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:43.472442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd18d6a-f170-4cbf-9c06-bc5094fff018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:43.473357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd18d6a-f170-4cbf-9c06-bc5094fff018 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:43.473860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd18d6a-f170-4cbf-9c06-bc5094fff018 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:43.474350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd18d6a-f170-4cbf-9c06-bc5094fff018 None admin] PATH_INFO: `/v3/projects/e6449d9e47674424b968cf98dfa19948` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:43.476407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd18d6a-f170-4cbf-9c06-bc5094fff018 None admin] RBAC: Authorizing `identity:get_project(project_id=e6449d9e47674424b968cf98dfa19948)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:43.479010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd18d6a-f170-4cbf-9c06-bc5094fff018 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:43.481450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 47/95] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:43 2022] GET /identity/v3/projects/e6449d9e47674424b968cf98dfa19948 => generated 312 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:43.484451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 49/96] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:43 2022] GET /identity/v3/role_assignments?user.id=05a7b57c5cfe4d368e6c200fc07ecf2d&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:43.492415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd18d6a-f170-4cbf-9c06-bc5094fff018 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:43.500936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:43.646876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd18d6a-f170-4cbf-9c06-bc5094fff018 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:43.706611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa78a93-6754-4beb-a011-13b7ae6ce3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:43.707251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa78a93-6754-4beb-a011-13b7ae6ce3fc None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:43.707542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa78a93-6754-4beb-a011-13b7ae6ce3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:43.707835 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa78a93-6754-4beb-a011-13b7ae6ce3fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:43.708648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa78a93-6754-4beb-a011-13b7ae6ce3fc None admin] RBAC: Adding query filter params (role.id=c4fc73849d664b2588bd2de7372aa839, scope.project.id=e6449d9e47674424b968cf98dfa19948, user.id=05a7b57c5cfe4d368e6c200fc07ecf2d) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:43.708965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa78a93-6754-4beb-a011-13b7ae6ce3fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:43.711213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa78a93-6754-4beb-a011-13b7ae6ce3fc None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:43.721442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 48/97] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:43 2022] GET /identity/v3/role_assignments?user.id=05a7b57c5cfe4d368e6c200fc07ecf2d&scope.project.id=e6449d9e47674424b968cf98dfa19948&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:45.633094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccb8ec-ef62-4862-ac77-64a69366d903 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:45.633633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccb8ec-ef62-4862-ac77-64a69366d903 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:45.633633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccb8ec-ef62-4862-ac77-64a69366d903 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:45.635228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 50/98] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:45.640872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884a1d5-848e-4713-b143-9f1d8201c765 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:45.641354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884a1d5-848e-4713-b143-9f1d8201c765 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:45.641499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884a1d5-848e-4713-b143-9f1d8201c765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:45.662406 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c958-3f49-4446-ac79-e7edd86c4efe None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:45.663273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c958-3f49-4446-ac79-e7edd86c4efe None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:45.663713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c958-3f49-4446-ac79-e7edd86c4efe None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:45.665361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 51/99] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:45.666803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2884a1d5-848e-4713-b143-9f1d8201c765 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:45.669340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2884a1d5-848e-4713-b143-9f1d8201c765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:45.683027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a813e-425f-423a-850c-c7f8f2e01c15 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:45.683317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a813e-425f-423a-850c-c7f8f2e01c15 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:45.683607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a813e-425f-423a-850c-c7f8f2e01c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:45.722616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-011a813e-425f-423a-850c-c7f8f2e01c15 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:45.723828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-011a813e-425f-423a-850c-c7f8f2e01c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:45.813769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:45.830988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 49/100] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:45.833961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 52/101] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:08:45.870300 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884a1d5-848e-4713-b143-9f1d8201c765 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:45.875025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a813e-425f-423a-850c-c7f8f2e01c15 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:46.029688 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:46.043792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a813e-425f-423a-850c-c7f8f2e01c15 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:46.097888 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc0f3b-8365-48c9-8e08-83f51d3f94ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:46.098549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc0f3b-8365-48c9-8e08-83f51d3f94ef None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.098835 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc0f3b-8365-48c9-8e08-83f51d3f94ef None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.099116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc0f3b-8365-48c9-8e08-83f51d3f94ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.101827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc0f3b-8365-48c9-8e08-83f51d3f94ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:46.103307 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc0f3b-8365-48c9-8e08-83f51d3f94ef None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:46.104922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 53/102] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:46.111012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc0f3b-8365-48c9-8e08-83f51d3f94ef None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:46.115255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884a1d5-848e-4713-b143-9f1d8201c765 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:46.166953 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006caaf8-5d0f-4f1c-ab46-e5a8f2245243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:46.167616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006caaf8-5d0f-4f1c-ab46-e5a8f2245243 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.167907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006caaf8-5d0f-4f1c-ab46-e5a8f2245243 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.168192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006caaf8-5d0f-4f1c-ab46-e5a8f2245243 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.181609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006caaf8-5d0f-4f1c-ab46-e5a8f2245243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-006caaf8-5d0f-4f1c-ab46-e5a8f2245243 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:46.189334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:46.190248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 50/103] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:46.191056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14443af8-e534-47de-88b8-4d120c46149a None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.191744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14443af8-e534-47de-88b8-4d120c46149a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.191744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14443af8-e534-47de-88b8-4d120c46149a None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.196816 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 51/104] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:46.197710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14443af8-e534-47de-88b8-4d120c46149a None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:46.294173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc0f3b-8365-48c9-8e08-83f51d3f94ef None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:46.333915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8ed19-df61-4da4-8271-1e51a08b5f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:46.336058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:46.336181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8ed19-df61-4da4-8271-1e51a08b5f8e None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.336381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8ed19-df61-4da4-8271-1e51a08b5f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.336664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8ed19-df61-4da4-8271-1e51a08b5f8e None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.337528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8ed19-df61-4da4-8271-1e51a08b5f8e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:46.339010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8ed19-df61-4da4-8271-1e51a08b5f8e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:46.407357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14443af8-e534-47de-88b8-4d120c46149a None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:46.424140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 54/105] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:08:46 2022] POST /identity/v3/projects => generated 302 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:08:46.426131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a263087-c756-40fb-8539-4972e3bf071e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.430708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a263087-c756-40fb-8539-4972e3bf071e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.430708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a263087-c756-40fb-8539-4972e3bf071e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.448495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f828780-15ab-4c9d-a0b5-d69bd61280d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:46.449217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f828780-15ab-4c9d-a0b5-d69bd61280d8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.449539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f828780-15ab-4c9d-a0b5-d69bd61280d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.449884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f828780-15ab-4c9d-a0b5-d69bd61280d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.450634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f828780-15ab-4c9d-a0b5-d69bd61280d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:46.451002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f828780-15ab-4c9d-a0b5-d69bd61280d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:46.452454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f828780-15ab-4c9d-a0b5-d69bd61280d8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:46.455872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3a263087-c756-40fb-8539-4972e3bf071e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:46.457410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3a263087-c756-40fb-8539-4972e3bf071e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:46.462777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 52/106] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:08:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:46.469358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98004ec-dfc6-4fd7-a1ed-47a2626a2ea6 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.469358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98004ec-dfc6-4fd7-a1ed-47a2626a2ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.469358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98004ec-dfc6-4fd7-a1ed-47a2626a2ea6 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.469358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 53/107] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:46.470603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98004ec-dfc6-4fd7-a1ed-47a2626a2ea6 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:46.473629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:46.562459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 55/108] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:46.564303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc596-adcf-465a-a710-64bf7337c313 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.564817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc596-adcf-465a-a710-64bf7337c313 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.565352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc596-adcf-465a-a710-64bf7337c313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.589743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5d5cc596-adcf-465a-a710-64bf7337c313 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:46.591763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5d5cc596-adcf-465a-a710-64bf7337c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:46.617203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98004ec-dfc6-4fd7-a1ed-47a2626a2ea6 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:46.640747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:46.665054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d172-22cb-4368-b94e-83389b1d7359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:46.665738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d172-22cb-4368-b94e-83389b1d7359 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.665738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d172-22cb-4368-b94e-83389b1d7359 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.666040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d172-22cb-4368-b94e-83389b1d7359 None admin] PATH_INFO: `/v3/users/fb8360e10f594311b18d2d4276f01ee9` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.687866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847d172-22cb-4368-b94e-83389b1d7359 None admin] RBAC: Authorizing `identity:get_user(user_id=fb8360e10f594311b18d2d4276f01ee9)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:46.688627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 56/109] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:46.689388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847d172-22cb-4368-b94e-83389b1d7359 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:46.691416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 54/110] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:46 2022] GET /identity/v3/users/fb8360e10f594311b18d2d4276f01ee9 => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:46.692500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cc596-adcf-465a-a710-64bf7337c313 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:46.700789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d172-22cb-4368-b94e-83389b1d7359 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:46.793333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:46.826317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cc596-adcf-465a-a710-64bf7337c313 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:46.848890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d172-22cb-4368-b94e-83389b1d7359 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:46.871939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795fd41-0ae6-454a-9287-0b19073aefd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:46.872609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795fd41-0ae6-454a-9287-0b19073aefd1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.872899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795fd41-0ae6-454a-9287-0b19073aefd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.873183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795fd41-0ae6-454a-9287-0b19073aefd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.885126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795fd41-0ae6-454a-9287-0b19073aefd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8795fd41-0ae6-454a-9287-0b19073aefd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:46.890115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:46.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:08:46.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 57/111] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:08:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:46.897222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795fd41-0ae6-454a-9287-0b19073aefd1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:46.905662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c77d0-9787-4e8e-b427-334aa525fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:46.906661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c77d0-9787-4e8e-b427-334aa525fc8a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:46.907668 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c77d0-9787-4e8e-b427-334aa525fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:46.908181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c77d0-9787-4e8e-b427-334aa525fc8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:46.962805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:46.988770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c77d0-9787-4e8e-b427-334aa525fc8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:46.990357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c77d0-9787-4e8e-b427-334aa525fc8a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:47.016025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795fd41-0ae6-454a-9287-0b19073aefd1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:47.049515 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:47.056488 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb55e66-4109-485b-a662-4c17f01f1d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:47.057885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb55e66-4109-485b-a662-4c17f01f1d66 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.062813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb55e66-4109-485b-a662-4c17f01f1d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.062813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb55e66-4109-485b-a662-4c17f01f1d66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.072972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb55e66-4109-485b-a662-4c17f01f1d66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4fb55e66-4109-485b-a662-4c17f01f1d66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:47.080189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:47.081285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:47.082086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:08:47.082995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 58/112] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:47.083918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c8702-a7c7-418a-90af-c003aada543e None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.084370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c8702-a7c7-418a-90af-c003aada543e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.084660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c8702-a7c7-418a-90af-c003aada543e None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.090461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 59/113] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:47.090461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c8702-a7c7-418a-90af-c003aada543e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:47.106618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 55/114] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:47.107906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882257de-c858-4bff-9e78-52eca0d6513f None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.108198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882257de-c858-4bff-9e78-52eca0d6513f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.108479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882257de-c858-4bff-9e78-52eca0d6513f None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.109910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 56/115] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:47.110448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882257de-c858-4bff-9e78-52eca0d6513f None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:47.199001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:47.219231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c8702-a7c7-418a-90af-c003aada543e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:47.222255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882257de-c858-4bff-9e78-52eca0d6513f None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:47.259974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afa195-8655-4fac-a472-81873882744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:47.260900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afa195-8655-4fac-a472-81873882744a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.261192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afa195-8655-4fac-a472-81873882744a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.261489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c5430-59e4-49db-9b48-31a079a80f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:47.261641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afa195-8655-4fac-a472-81873882744a None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.261859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c5430-59e4-49db-9b48-31a079a80f81 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.262146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c5430-59e4-49db-9b48-31a079a80f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.262240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afa195-8655-4fac-a472-81873882744a None admin] RBAC: Adding query filter params (name=member) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:47.262426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c5430-59e4-49db-9b48-31a079a80f81 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.262533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afa195-8655-4fac-a472-81873882744a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:47.263240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c5430-59e4-49db-9b48-31a079a80f81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:47.263545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c5430-59e4-49db-9b48-31a079a80f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:47.263945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afa195-8655-4fac-a472-81873882744a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:47.264940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c5430-59e4-49db-9b48-31a079a80f81 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:47.274438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 57/116] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:08:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:47.279929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 60/117] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:47.279929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c5430-59e4-49db-9b48-31a079a80f81 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:47.280331 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ced115-8968-472e-803e-21f163b28584 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.280618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ced115-8968-472e-803e-21f163b28584 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.280908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ced115-8968-472e-803e-21f163b28584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.299995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-11ced115-8968-472e-803e-21f163b28584 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:47.301219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-11ced115-8968-472e-803e-21f163b28584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:47.339534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:47.377248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 58/118] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:47.378874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb16f-7cb2-4630-aa92-d998f89e2f67 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.379292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb16f-7cb2-4630-aa92-d998f89e2f67 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.379573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb16f-7cb2-4630-aa92-d998f89e2f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.402855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c5430-59e4-49db-9b48-31a079a80f81 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:47.410963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b66eb16f-7cb2-4630-aa92-d998f89e2f67 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:47.412808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b66eb16f-7cb2-4630-aa92-d998f89e2f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:47.440751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eaf54-387f-42ef-99ea-7d45dbe90d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:47.441453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eaf54-387f-42ef-99ea-7d45dbe90d08 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.441751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eaf54-387f-42ef-99ea-7d45dbe90d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.442035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eaf54-387f-42ef-99ea-7d45dbe90d08 None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.443214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eaf54-387f-42ef-99ea-7d45dbe90d08 None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:47.445421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eaf54-387f-42ef-99ea-7d45dbe90d08 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:47.447128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 61/119] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:47 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:47.447932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eaf54-387f-42ef-99ea-7d45dbe90d08 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:47.485601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:47.490835 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 59/120] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:47.491845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66eb16f-7cb2-4630-aa92-d998f89e2f67 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:47.578409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eaf54-387f-42ef-99ea-7d45dbe90d08 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:47.595985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66eb16f-7cb2-4630-aa92-d998f89e2f67 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:47.605409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:47.617538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ddc3-6272-421a-9718-30c46e52075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:47.635738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ddc3-6272-421a-9718-30c46e52075e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.635738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ddc3-6272-421a-9718-30c46e52075e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.635738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923ddc3-6272-421a-9718-30c46e52075e None admin] PATH_INFO: `/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.635738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923ddc3-6272-421a-9718-30c46e52075e None admin] RBAC: Authorizing `identity:get_user(user_id=05a7b57c5cfe4d368e6c200fc07ecf2d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:47.635738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923ddc3-6272-421a-9718-30c46e52075e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:47.635738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 62/121] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:47 2022] GET /identity/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:47.636129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ddc3-6272-421a-9718-30c46e52075e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:47.637677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edb13e-2f28-4107-951b-ed05e440afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:47.638293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edb13e-2f28-4107-951b-ed05e440afb0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.638580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edb13e-2f28-4107-951b-ed05e440afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.638858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edb13e-2f28-4107-951b-ed05e440afb0 None admin] PATH_INFO: `/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.640023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edb13e-2f28-4107-951b-ed05e440afb0 None admin] RBAC: Authorizing `identity:get_user(user_id=b42a77304cdd4ceda1f777fdc9b5ccf9)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:47.641792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edb13e-2f28-4107-951b-ed05e440afb0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:47.648503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 60/122] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:47 2022] GET /identity/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:47.648503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edb13e-2f28-4107-951b-ed05e440afb0 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:47.719910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:47.746799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923ddc3-6272-421a-9718-30c46e52075e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:47.757636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edb13e-2f28-4107-951b-ed05e440afb0 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:47.795574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b9263-ce39-40e8-abce-241dadc636c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:47.796520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b9263-ce39-40e8-abce-241dadc636c9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.797044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b9263-ce39-40e8-abce-241dadc636c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.797528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b9263-ce39-40e8-abce-241dadc636c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.797642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e178138-4c78-478a-bcbc-0a3c5f5d5d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:47.798252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e178138-4c78-478a-bcbc-0a3c5f5d5d8b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:47.798538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e178138-4c78-478a-bcbc-0a3c5f5d5d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:47.798829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e178138-4c78-478a-bcbc-0a3c5f5d5d8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:47.799637 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e178138-4c78-478a-bcbc-0a3c5f5d5d8b None admin] RBAC: Adding query filter params (role.id=8e94dcc6eec7482fab6c8aba670a4b6c, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3, user.id=fb8360e10f594311b18d2d4276f01ee9) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:47.799954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e178138-4c78-478a-bcbc-0a3c5f5d5d8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:47.801461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e178138-4c78-478a-bcbc-0a3c5f5d5d8b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:47.807958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 61/123] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:47 2022] GET /identity/v3/role_assignments?user.id=fb8360e10f594311b18d2d4276f01ee9&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=8e94dcc6eec7482fab6c8aba670a4b6c => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:47.808604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e178138-4c78-478a-bcbc-0a3c5f5d5d8b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:47.822258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b9263-ce39-40e8-abce-241dadc636c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-458b9263-ce39-40e8-abce-241dadc636c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:47.828215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:47.829148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:08:47.829933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 63/124] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:08:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:47.830745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b9263-ce39-40e8-abce-241dadc636c9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:47.901077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:47.962161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b9263-ce39-40e8-abce-241dadc636c9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:47.967756 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e178138-4c78-478a-bcbc-0a3c5f5d5d8b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:48.007813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a2c62-74d2-4bae-b71a-dbda267475ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:48.008494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a2c62-74d2-4bae-b71a-dbda267475ee None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:48.008787 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a2c62-74d2-4bae-b71a-dbda267475ee None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:48.009082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a2c62-74d2-4bae-b71a-dbda267475ee None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:48.010377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a2c62-74d2-4bae-b71a-dbda267475ee None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:48.011769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a2c62-74d2-4bae-b71a-dbda267475ee None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:48.013527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 64/125] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:47 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:48.014737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a2c62-74d2-4bae-b71a-dbda267475ee None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:48.015169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:48.026071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703d0c8-84cd-4c9b-a0fb-77d87a0870a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:48.030785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703d0c8-84cd-4c9b-a0fb-77d87a0870a0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:48.030785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703d0c8-84cd-4c9b-a0fb-77d87a0870a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:48.030785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703d0c8-84cd-4c9b-a0fb-77d87a0870a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:48.043023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703d0c8-84cd-4c9b-a0fb-77d87a0870a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-4703d0c8-84cd-4c9b-a0fb-77d87a0870a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:48.050883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:48.051809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:48.052643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 62/126] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:48.057341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703d0c8-84cd-4c9b-a0fb-77d87a0870a0 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:48.133585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a2c62-74d2-4bae-b71a-dbda267475ee None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:48.152508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:48.171342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedaacc-bf04-486b-a87b-85792cfc590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:48.171562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703d0c8-84cd-4c9b-a0fb-77d87a0870a0 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:48.172472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedaacc-bf04-486b-a87b-85792cfc590d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:48.172973 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedaacc-bf04-486b-a87b-85792cfc590d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:48.173519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedaacc-bf04-486b-a87b-85792cfc590d None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:48.175276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedaacc-bf04-486b-a87b-85792cfc590d None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:48.176987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedaacc-bf04-486b-a87b-85792cfc590d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:48.180490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 65/127] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:48 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:48.181765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedaacc-bf04-486b-a87b-85792cfc590d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:48.214165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed26b58-9f33-457a-b72d-83038c7e4041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:48.214843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed26b58-9f33-457a-b72d-83038c7e4041 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:48.215144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed26b58-9f33-457a-b72d-83038c7e4041 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:48.215433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed26b58-9f33-457a-b72d-83038c7e4041 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:48.216172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed26b58-9f33-457a-b72d-83038c7e4041 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:48.216490 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed26b58-9f33-457a-b72d-83038c7e4041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:48.217921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed26b58-9f33-457a-b72d-83038c7e4041 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:48.227735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 63/128] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:08:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:48.232679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed26b58-9f33-457a-b72d-83038c7e4041 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:48.294121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedaacc-bf04-486b-a87b-85792cfc590d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:48.297247 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:48.335532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb4ff7-43fe-4858-bef4-e38ab9f2e2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:48.336230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb4ff7-43fe-4858-bef4-e38ab9f2e2bd None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:48.336526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb4ff7-43fe-4858-bef4-e38ab9f2e2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:48.336815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb4ff7-43fe-4858-bef4-e38ab9f2e2bd None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/users/05a7b57c5cfe4d368e6c200fc07ecf2d/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:48.338972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb4ff7-43fe-4858-bef4-e38ab9f2e2bd None admin] RBAC: Authorizing `identity:create_grant(project_id=e646eb9882194d2f8e2bf2ec959ecff3, user_id=05a7b57c5cfe4d368e6c200fc07ecf2d, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:48.340526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb4ff7-43fe-4858-bef4-e38ab9f2e2bd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:48.355962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed26b58-9f33-457a-b72d-83038c7e4041 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:48.397319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333f12b-f6e6-4320-a6ab-a66be5cf2eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:48.398932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333f12b-f6e6-4320-a6ab-a66be5cf2eec None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:48.398932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333f12b-f6e6-4320-a6ab-a66be5cf2eec None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:48.398932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333f12b-f6e6-4320-a6ab-a66be5cf2eec None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:48.399594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333f12b-f6e6-4320-a6ab-a66be5cf2eec None admin] RBAC: Adding query filter params (name=member) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:48.399911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333f12b-f6e6-4320-a6ab-a66be5cf2eec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:48.401363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333f12b-f6e6-4320-a6ab-a66be5cf2eec None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:48.415734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 64/129] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:08:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:48.415734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333f12b-f6e6-4320-a6ab-a66be5cf2eec None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:48.436154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 66/130] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:08:48 2022] PUT /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/users/05a7b57c5cfe4d368e6c200fc07ecf2d/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:08:48.442906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb4ff7-43fe-4858-bef4-e38ab9f2e2bd None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:48.506334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:48.587316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb4ff7-43fe-4858-bef4-e38ab9f2e2bd None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:48.590782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333f12b-f6e6-4320-a6ab-a66be5cf2eec None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:48.636268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:48.638397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde61efe-b235-4fda-9715-54cf271d2507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:48.639309 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde61efe-b235-4fda-9715-54cf271d2507 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:48.639822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde61efe-b235-4fda-9715-54cf271d2507 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:48.640561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde61efe-b235-4fda-9715-54cf271d2507 None admin] PATH_INFO: `/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:48.642606 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde61efe-b235-4fda-9715-54cf271d2507 None admin] RBAC: Authorizing `identity:get_user(user_id=05a7b57c5cfe4d368e6c200fc07ecf2d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:48.644711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde61efe-b235-4fda-9715-54cf271d2507 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:48.648418 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 67/131] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:48 2022] GET /identity/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:48.653141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde61efe-b235-4fda-9715-54cf271d2507 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:48.660730 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0b668-f1d7-48d4-aa75-a6717e1355dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:48.661711 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b668-f1d7-48d4-aa75-a6717e1355dd None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:48.661817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b668-f1d7-48d4-aa75-a6717e1355dd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:48.662673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b668-f1d7-48d4-aa75-a6717e1355dd None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/users/b42a77304cdd4ceda1f777fdc9b5ccf9/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:48.665339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0b668-f1d7-48d4-aa75-a6717e1355dd None admin] RBAC: Authorizing `identity:create_grant(project_id=e646eb9882194d2f8e2bf2ec959ecff3, user_id=b42a77304cdd4ceda1f777fdc9b5ccf9, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:48.667063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0b668-f1d7-48d4-aa75-a6717e1355dd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:48.761967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde61efe-b235-4fda-9715-54cf271d2507 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:48.781030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:48.782696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 65/132] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:08:48 2022] PUT /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/users/b42a77304cdd4ceda1f777fdc9b5ccf9/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:08:48.783368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0b668-f1d7-48d4-aa75-a6717e1355dd None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:48.819870 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489d713-d45b-419a-8377-6ead0384da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:48.820569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489d713-d45b-419a-8377-6ead0384da79 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:48.820867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489d713-d45b-419a-8377-6ead0384da79 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:48.821159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489d713-d45b-419a-8377-6ead0384da79 None admin] PATH_INFO: `/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:48.826260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489d713-d45b-419a-8377-6ead0384da79 None admin] RBAC: Authorizing `identity:get_user(user_id=05a7b57c5cfe4d368e6c200fc07ecf2d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:48.827801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489d713-d45b-419a-8377-6ead0384da79 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:48.829754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 68/133] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:48 2022] GET /identity/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:48.830993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeeb4d-9dce-4c7d-b0c3-fd79e7a724dd None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:48.831306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeeb4d-9dce-4c7d-b0c3-fd79e7a724dd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:48.831580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeeb4d-9dce-4c7d-b0c3-fd79e7a724dd None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:48.832895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 69/134] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:48.838363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeeeb4d-9dce-4c7d-b0c3-fd79e7a724dd None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:48.961078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0b668-f1d7-48d4-aa75-a6717e1355dd None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:49.000121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574d338-e3e8-4b6d-aa13-8fc6187493ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:49.000777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d338-e3e8-4b6d-aa13-8fc6187493ed None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:49.001055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d338-e3e8-4b6d-aa13-8fc6187493ed None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:49.001928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d338-e3e8-4b6d-aa13-8fc6187493ed None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:49.003587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574d338-e3e8-4b6d-aa13-8fc6187493ed None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:49.004890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574d338-e3e8-4b6d-aa13-8fc6187493ed None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:49.007576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 66/135] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:48 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:49.008913 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09f26e-e92e-4e95-92b9-210a48634e27 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:49.009169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09f26e-e92e-4e95-92b9-210a48634e27 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:49.009484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09f26e-e92e-4e95-92b9-210a48634e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:49.030904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-7f09f26e-e92e-4e95-92b9-210a48634e27 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:49.032198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7f09f26e-e92e-4e95-92b9-210a48634e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:49.037040 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeeeb4d-9dce-4c7d-b0c3-fd79e7a724dd None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:49.075502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01ff09-161b-45f5-988b-61f07bd4987a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:49.076646 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01ff09-161b-45f5-988b-61f07bd4987a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:49.077167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01ff09-161b-45f5-988b-61f07bd4987a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:49.077679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01ff09-161b-45f5-988b-61f07bd4987a None admin] PATH_INFO: `/v3/projects/e6449d9e47674424b968cf98dfa19948` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:49.079109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01ff09-161b-45f5-988b-61f07bd4987a None admin] RBAC: Authorizing `identity:get_project(project_id=e6449d9e47674424b968cf98dfa19948)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:49.080859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01ff09-161b-45f5-988b-61f07bd4987a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:49.081666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:49.083199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 70/136] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:48 2022] GET /identity/v3/projects/e6449d9e47674424b968cf98dfa19948 => generated 312 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:49.084242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01ff09-161b-45f5-988b-61f07bd4987a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:49.118180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 67/137] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:49.119053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09f26e-e92e-4e95-92b9-210a48634e27 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:49.190462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:49.261520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09f26e-e92e-4e95-92b9-210a48634e27 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:49.283098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01ff09-161b-45f5-988b-61f07bd4987a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:49.301844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034aa2f-6737-473c-8736-0c52583c8787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:49.303583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034aa2f-6737-473c-8736-0c52583c8787 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:49.304347 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034aa2f-6737-473c-8736-0c52583c8787 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:49.304857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034aa2f-6737-473c-8736-0c52583c8787 None admin] PATH_INFO: `/v3/projects/e6449d9e47674424b968cf98dfa19948/users/05a7b57c5cfe4d368e6c200fc07ecf2d/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:49.310843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034aa2f-6737-473c-8736-0c52583c8787 None admin] RBAC: Authorizing `identity:create_grant(project_id=e6449d9e47674424b968cf98dfa19948, user_id=05a7b57c5cfe4d368e6c200fc07ecf2d, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:49.314480 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034aa2f-6737-473c-8736-0c52583c8787 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:49.339355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca7284-faa5-4503-86e8-5afa8e8e454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:49.340459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca7284-faa5-4503-86e8-5afa8e8e454a None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:49.340745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca7284-faa5-4503-86e8-5afa8e8e454a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:49.341024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca7284-faa5-4503-86e8-5afa8e8e454a None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/users/05a7b57c5cfe4d368e6c200fc07ecf2d/roles/c7b4bfc82f3b46cfaa10773dad7cd03a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:49.343396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca7284-faa5-4503-86e8-5afa8e8e454a None admin] RBAC: Authorizing `identity:create_grant(project_id=e646eb9882194d2f8e2bf2ec959ecff3, user_id=05a7b57c5cfe4d368e6c200fc07ecf2d, role_id=c7b4bfc82f3b46cfaa10773dad7cd03a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:49.344978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca7284-faa5-4503-86e8-5afa8e8e454a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:49.387182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 68/138] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:08:49 2022] PUT /identity/v3/projects/e6449d9e47674424b968cf98dfa19948/users/05a7b57c5cfe4d368e6c200fc07ecf2d/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:08:49.388053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034aa2f-6737-473c-8736-0c52583c8787 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:49.415959 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 71/139] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:08:49 2022] PUT /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/users/05a7b57c5cfe4d368e6c200fc07ecf2d/roles/c7b4bfc82f3b46cfaa10773dad7cd03a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:08:49.437041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:49.524451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034aa2f-6737-473c-8736-0c52583c8787 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:49.578174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7af6a0-98b8-42eb-aa64-b28e87df5279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:49.579283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7af6a0-98b8-42eb-aa64-b28e87df5279 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:49.579782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7af6a0-98b8-42eb-aa64-b28e87df5279 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:49.580266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7af6a0-98b8-42eb-aa64-b28e87df5279 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:49.582069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7af6a0-98b8-42eb-aa64-b28e87df5279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:49.583823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7af6a0-98b8-42eb-aa64-b28e87df5279 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:49.586029 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 69/140] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:49.592163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca7284-faa5-4503-86e8-5afa8e8e454a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:49.751369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca7284-faa5-4503-86e8-5afa8e8e454a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:49.766691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:49.798268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494e8ea-9cf2-4878-a28c-a88a4a1b0896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:49.798840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494e8ea-9cf2-4878-a28c-a88a4a1b0896 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:49.799133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494e8ea-9cf2-4878-a28c-a88a4a1b0896 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:49.799416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494e8ea-9cf2-4878-a28c-a88a4a1b0896 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:49.800161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494e8ea-9cf2-4878-a28c-a88a4a1b0896 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:49.801684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494e8ea-9cf2-4878-a28c-a88a4a1b0896 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:49.884661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 72/141] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:08:49 2022] POST /identity/v3/users => generated 283 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:08:50.308343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4ca68-2cc3-45d8-834b-bca7cc896a05 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:50.308343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4ca68-2cc3-45d8-834b-bca7cc896a05 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:50.308343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4ca68-2cc3-45d8-834b-bca7cc896a05 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:50.308343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 70/142] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:50.315160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69cad-23de-4f06-af96-aa494afd6bd9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:50.315906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69cad-23de-4f06-af96-aa494afd6bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:50.316390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69cad-23de-4f06-af96-aa494afd6bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:50.344883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-31c69cad-23de-4f06-af96-aa494afd6bd9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:50.346855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-31c69cad-23de-4f06-af96-aa494afd6bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:50.426721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 73/143] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:50.449144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4ca68-2cc3-45d8-834b-bca7cc896a05 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:50.587309 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4ca68-2cc3-45d8-834b-bca7cc896a05 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:50.589035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:50.629090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22516ab9-3bd7-4099-8a74-4d9c1820a90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:50.629784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22516ab9-3bd7-4099-8a74-4d9c1820a90f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:50.630076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22516ab9-3bd7-4099-8a74-4d9c1820a90f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:50.630362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22516ab9-3bd7-4099-8a74-4d9c1820a90f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:50.650190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22516ab9-3bd7-4099-8a74-4d9c1820a90f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-22516ab9-3bd7-4099-8a74-4d9c1820a90f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:50.655171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:50.656025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:50.656777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 71/144] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:50.663210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c69cad-23de-4f06-af96-aa494afd6bd9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:50.869889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:50.877058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c69cad-23de-4f06-af96-aa494afd6bd9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:50.927400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404714d-ce08-4298-a6a6-74787f8ff05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:50.928328 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404714d-ce08-4298-a6a6-74787f8ff05d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:50.928635 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404714d-ce08-4298-a6a6-74787f8ff05d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:50.929005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404714d-ce08-4298-a6a6-74787f8ff05d None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:50.929789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404714d-ce08-4298-a6a6-74787f8ff05d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:50.930103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404714d-ce08-4298-a6a6-74787f8ff05d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:50.931524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404714d-ce08-4298-a6a6-74787f8ff05d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:50.946616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 74/145] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:08:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:50.952591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22516ab9-3bd7-4099-8a74-4d9c1820a90f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:51.129248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22516ab9-3bd7-4099-8a74-4d9c1820a90f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:51.156447 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700db98e-8d89-4ca3-a815-2696da93e308 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:51.157093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700db98e-8d89-4ca3-a815-2696da93e308 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:51.157609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700db98e-8d89-4ca3-a815-2696da93e308 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:51.159160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 75/146] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:51.167151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8e29b-83e1-4b1c-8f25-e3a39da9a920 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:51.167714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8e29b-83e1-4b1c-8f25-e3a39da9a920 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:51.168568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8e29b-83e1-4b1c-8f25-e3a39da9a920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:51.191088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acb3f3-7784-488a-b2fb-0f57fa008e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:51.191088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acb3f3-7784-488a-b2fb-0f57fa008e63 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:51.191088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acb3f3-7784-488a-b2fb-0f57fa008e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:51.191387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acb3f3-7784-488a-b2fb-0f57fa008e63 None admin] PATH_INFO: `/v3/users/fb8360e10f594311b18d2d4276f01ee9` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:51.192664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-17d8e29b-83e1-4b1c-8f25-e3a39da9a920 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:51.195247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-17d8e29b-83e1-4b1c-8f25-e3a39da9a920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:51.198688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acb3f3-7784-488a-b2fb-0f57fa008e63 None admin] RBAC: Authorizing `identity:get_user(user_id=fb8360e10f594311b18d2d4276f01ee9)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:51.200595 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acb3f3-7784-488a-b2fb-0f57fa008e63 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:51.208614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 72/147] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:50 2022] GET /identity/v3/users/fb8360e10f594311b18d2d4276f01ee9 => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:51.208614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acb3f3-7784-488a-b2fb-0f57fa008e63 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:51.273271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:51.294174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 76/148] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:08:51.321817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8e29b-83e1-4b1c-8f25-e3a39da9a920 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:51.431199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:51.434368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acb3f3-7784-488a-b2fb-0f57fa008e63 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:51.450085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8e29b-83e1-4b1c-8f25-e3a39da9a920 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:51.494135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1673b2-6c86-474f-b7f0-1f93feaeabdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:51.494832 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1673b2-6c86-474f-b7f0-1f93feaeabdc None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:51.495126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1673b2-6c86-474f-b7f0-1f93feaeabdc None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:51.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1673b2-6c86-474f-b7f0-1f93feaeabdc None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:51.496699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1673b2-6c86-474f-b7f0-1f93feaeabdc None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:51.497561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d73cf-7971-4bc3-a419-067821e3af88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:51.498225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1673b2-6c86-474f-b7f0-1f93feaeabdc None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:51.498572 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d73cf-7971-4bc3-a419-067821e3af88 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:51.499037 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d73cf-7971-4bc3-a419-067821e3af88 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:51.499457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d73cf-7971-4bc3-a419-067821e3af88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:51.499934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 73/149] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:51 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:51.506535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1673b2-6c86-474f-b7f0-1f93feaeabdc None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:51.513599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d73cf-7971-4bc3-a419-067821e3af88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ec7d73cf-7971-4bc3-a419-067821e3af88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:51.519386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:51.520421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:51.523221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:08:51.525222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 77/150] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:08:51.531261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d73cf-7971-4bc3-a419-067821e3af88 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:51.648476 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d73cf-7971-4bc3-a419-067821e3af88 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:51.659834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1673b2-6c86-474f-b7f0-1f93feaeabdc None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:51.663535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:51.687879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b274c-1242-43ef-8e51-a5b8858b7224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:51.688471 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b274c-1242-43ef-8e51-a5b8858b7224 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:51.688755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b274c-1242-43ef-8e51-a5b8858b7224 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:51.689031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b274c-1242-43ef-8e51-a5b8858b7224 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:51.689800 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b274c-1242-43ef-8e51-a5b8858b7224 None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:51.690102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b274c-1242-43ef-8e51-a5b8858b7224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:51.691571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b274c-1242-43ef-8e51-a5b8858b7224 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:51.701641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 78/151] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:08:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:51.707091 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b274c-1242-43ef-8e51-a5b8858b7224 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:51.716191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f331a2-eb4f-4475-a18f-8c5b4e61a194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:51.716987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f331a2-eb4f-4475-a18f-8c5b4e61a194 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:51.717283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f331a2-eb4f-4475-a18f-8c5b4e61a194 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:51.717601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f331a2-eb4f-4475-a18f-8c5b4e61a194 None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/users/fb8360e10f594311b18d2d4276f01ee9/roles/8e94dcc6eec7482fab6c8aba670a4b6c` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:51.722806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f331a2-eb4f-4475-a18f-8c5b4e61a194 None admin] RBAC: Authorizing `identity:create_grant(project_id=e646eb9882194d2f8e2bf2ec959ecff3, user_id=fb8360e10f594311b18d2d4276f01ee9, role_id=8e94dcc6eec7482fab6c8aba670a4b6c)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:51.723505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:51.725083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f331a2-eb4f-4475-a18f-8c5b4e61a194 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:51.788882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 74/152] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:08:51 2022] PUT /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/users/fb8360e10f594311b18d2d4276f01ee9/roles/8e94dcc6eec7482fab6c8aba670a4b6c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:08:51.831116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b274c-1242-43ef-8e51-a5b8858b7224 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:51.871709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f48792-d73f-4e4d-b987-744d1570ad95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:51.872485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f48792-d73f-4e4d-b987-744d1570ad95 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:51.872670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f48792-d73f-4e4d-b987-744d1570ad95 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:51.872915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f48792-d73f-4e4d-b987-744d1570ad95 None admin] PATH_INFO: `/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:51.874147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f48792-d73f-4e4d-b987-744d1570ad95 None admin] RBAC: Authorizing `identity:get_user(user_id=05a7b57c5cfe4d368e6c200fc07ecf2d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:51.875760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f48792-d73f-4e4d-b987-744d1570ad95 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:51.877441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 79/153] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:51 2022] GET /identity/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:51.884105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f331a2-eb4f-4475-a18f-8c5b4e61a194 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:52.003180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f331a2-eb4f-4475-a18f-8c5b4e61a194 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:52.047467 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c9eee-7918-4dfe-84e6-7952048a5603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:52.048185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c9eee-7918-4dfe-84e6-7952048a5603 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:52.048471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c9eee-7918-4dfe-84e6-7952048a5603 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:52.048749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c9eee-7918-4dfe-84e6-7952048a5603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:52.088324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:52.134911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c9eee-7918-4dfe-84e6-7952048a5603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:52.136766 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c9eee-7918-4dfe-84e6-7952048a5603 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:52.270652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 75/154] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:52.277484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f48792-d73f-4e4d-b987-744d1570ad95 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:52.369675 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:52.393586 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f48792-d73f-4e4d-b987-744d1570ad95 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:52.418656 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7598bd-dd16-46d8-afb6-788af268c193 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:52.418962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7598bd-dd16-46d8-afb6-788af268c193 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:52.419215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7598bd-dd16-46d8-afb6-788af268c193 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:52.420731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 76/155] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:08:52.425598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a19a8c-5c4e-4957-9422-401145294042 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:52.425882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a19a8c-5c4e-4957-9422-401145294042 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:52.426159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a19a8c-5c4e-4957-9422-401145294042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:52.455564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b9a19a8c-5c4e-4957-9422-401145294042 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:52.456772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b9a19a8c-5c4e-4957-9422-401145294042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:52.461282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbad4e-c592-4ea9-91fc-e87988f7c35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:52.463384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbad4e-c592-4ea9-91fc-e87988f7c35e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:52.463384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbad4e-c592-4ea9-91fc-e87988f7c35e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:52.463384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbad4e-c592-4ea9-91fc-e87988f7c35e None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:52.464418 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbad4e-c592-4ea9-91fc-e87988f7c35e None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:52.465906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbad4e-c592-4ea9-91fc-e87988f7c35e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:52.467768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 80/156] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:52 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:52.481363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbad4e-c592-4ea9-91fc-e87988f7c35e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:52.562606 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:52.586842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 77/157] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:08:52.593148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbad4e-c592-4ea9-91fc-e87988f7c35e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:52.629541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fe57a-2ad8-4810-9ddf-4a600bc3c1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:52.630215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fe57a-2ad8-4810-9ddf-4a600bc3c1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:52.630502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fe57a-2ad8-4810-9ddf-4a600bc3c1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:52.631327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fe57a-2ad8-4810-9ddf-4a600bc3c1c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:52.632536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fe57a-2ad8-4810-9ddf-4a600bc3c1c7 None admin] RBAC: Adding query filter params (role.id=c4fc73849d664b2588bd2de7372aa839, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3, user.id=05a7b57c5cfe4d368e6c200fc07ecf2d) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:52.632855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fe57a-2ad8-4810-9ddf-4a600bc3c1c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:52.634396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fe57a-2ad8-4810-9ddf-4a600bc3c1c7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:52.639654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a19a8c-5c4e-4957-9422-401145294042 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:52.641890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 81/158] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:52 2022] GET /identity/v3/role_assignments?user.id=05a7b57c5cfe4d368e6c200fc07ecf2d&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:52.779659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a19a8c-5c4e-4957-9422-401145294042 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:52.817510 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ea91e-463f-43a6-955d-c9ec1510c436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:52.818947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ea91e-463f-43a6-955d-c9ec1510c436 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:52.819336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ea91e-463f-43a6-955d-c9ec1510c436 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:52.819644 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ea91e-463f-43a6-955d-c9ec1510c436 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:52.831508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ea91e-463f-43a6-955d-c9ec1510c436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-473ea91e-463f-43a6-955d-c9ec1510c436 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:52.837175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:52.838092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:52.838957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 78/159] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:08:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:08:52.844612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fe57a-2ad8-4810-9ddf-4a600bc3c1c7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:52.859079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:52.939373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515a305-5819-4f7f-ace9-9e284d5bdc6d None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:52.939694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515a305-5819-4f7f-ace9-9e284d5bdc6d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:52.939932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515a305-5819-4f7f-ace9-9e284d5bdc6d None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:52.941299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 79/160] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:52.946952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b15bff-2da3-404f-ae04-7b15db873d74 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:52.947230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b15bff-2da3-404f-ae04-7b15db873d74 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:52.947507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b15bff-2da3-404f-ae04-7b15db873d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:52.982638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-17b15bff-2da3-404f-ae04-7b15db873d74 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:52.984066 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-17b15bff-2da3-404f-ae04-7b15db873d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:53.004600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fe57a-2ad8-4810-9ddf-4a600bc3c1c7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:53.077609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:53.082637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606b9e2-58cc-4817-8b2d-5c6635edef0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:53.083566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b9e2-58cc-4817-8b2d-5c6635edef0c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:53.084664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b9e2-58cc-4817-8b2d-5c6635edef0c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:53.085183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b9e2-58cc-4817-8b2d-5c6635edef0c None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:53.086454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606b9e2-58cc-4817-8b2d-5c6635edef0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:53.087060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606b9e2-58cc-4817-8b2d-5c6635edef0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:53.092393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606b9e2-58cc-4817-8b2d-5c6635edef0c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:53.102637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 82/161] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:53.109730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606b9e2-58cc-4817-8b2d-5c6635edef0c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:53.123760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 80/162] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:08:53.176233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b15bff-2da3-404f-ae04-7b15db873d74 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:53.226701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:53.286451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606b9e2-58cc-4817-8b2d-5c6635edef0c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:53.343348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b15bff-2da3-404f-ae04-7b15db873d74 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:53.354059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c80ffc-c76f-4246-a562-a9e73a110461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:53.354743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c80ffc-c76f-4246-a562-a9e73a110461 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:53.355027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c80ffc-c76f-4246-a562-a9e73a110461 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:53.355306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c80ffc-c76f-4246-a562-a9e73a110461 None admin] PATH_INFO: `/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:53.356559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c80ffc-c76f-4246-a562-a9e73a110461 None admin] RBAC: Authorizing `identity:get_user(user_id=b42a77304cdd4ceda1f777fdc9b5ccf9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:53.359124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c80ffc-c76f-4246-a562-a9e73a110461 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:53.359867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 83/163] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:53 2022] GET /identity/v3/users/b42a77304cdd4ceda1f777fdc9b5ccf9 => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:53.366020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c80ffc-c76f-4246-a562-a9e73a110461 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:53.401002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62745a9-c883-410f-bcec-98c93cbb142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:53.402126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62745a9-c883-410f-bcec-98c93cbb142a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:53.402630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62745a9-c883-410f-bcec-98c93cbb142a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:53.403120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62745a9-c883-410f-bcec-98c93cbb142a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:53.415302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62745a9-c883-410f-bcec-98c93cbb142a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-d62745a9-c883-410f-bcec-98c93cbb142a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:53.420694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:53.421570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:08:53.422427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:53.424771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 81/164] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:08:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:08:53.427041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309597a1-4b9a-4620-b19b-b7ddec1b8e91 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:53.427748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309597a1-4b9a-4620-b19b-b7ddec1b8e91 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:53.428367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309597a1-4b9a-4620-b19b-b7ddec1b8e91 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:53.430333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 82/165] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:53.431331 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309597a1-4b9a-4620-b19b-b7ddec1b8e91 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:53.487486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c80ffc-c76f-4246-a562-a9e73a110461 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:53.494003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:53.532379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c85bc-643d-4464-89e0-e3ac11aed6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:53.533137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c85bc-643d-4464-89e0-e3ac11aed6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:53.533607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c85bc-643d-4464-89e0-e3ac11aed6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:53.533888 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c85bc-643d-4464-89e0-e3ac11aed6c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:53.551609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:53.610192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309597a1-4b9a-4620-b19b-b7ddec1b8e91 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:53.635776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c85bc-643d-4464-89e0-e3ac11aed6c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:53.637298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c85bc-643d-4464-89e0-e3ac11aed6c7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:53.686032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709d486-2f17-47d4-a406-84555c63ce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:53.686686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709d486-2f17-47d4-a406-84555c63ce3e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:53.686981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709d486-2f17-47d4-a406-84555c63ce3e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:53.687263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709d486-2f17-47d4-a406-84555c63ce3e None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:53.688008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709d486-2f17-47d4-a406-84555c63ce3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:53.688320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709d486-2f17-47d4-a406-84555c63ce3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:53.689758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709d486-2f17-47d4-a406-84555c63ce3e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:53.703936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 83/166] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:08:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:53.705200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757bc93-967e-45d9-8552-1386ea748011 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:53.705500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757bc93-967e-45d9-8552-1386ea748011 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:53.705794 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757bc93-967e-45d9-8552-1386ea748011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:53.738365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-5757bc93-967e-45d9-8552-1386ea748011 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:53.739683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-5757bc93-967e-45d9-8552-1386ea748011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:53.753047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 84/167] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:08:53.758077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c85bc-643d-4464-89e0-e3ac11aed6c7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:53.805783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:53.860084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 84/168] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:53.861041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757bc93-967e-45d9-8552-1386ea748011 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:53.875696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c85bc-643d-4464-89e0-e3ac11aed6c7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:53.921771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:53.922481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f654f-e81e-4205-864c-a07ff70eaa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:53.923436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f654f-e81e-4205-864c-a07ff70eaa43 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:53.924012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f654f-e81e-4205-864c-a07ff70eaa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:53.924510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f654f-e81e-4205-864c-a07ff70eaa43 None admin] PATH_INFO: `/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:53.926221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f654f-e81e-4205-864c-a07ff70eaa43 None admin] RBAC: Authorizing `identity:get_user(user_id=05a7b57c5cfe4d368e6c200fc07ecf2d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:53.928186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f654f-e81e-4205-864c-a07ff70eaa43 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:53.937657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 85/169] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:53 2022] GET /identity/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:53.938968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f654f-e81e-4205-864c-a07ff70eaa43 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:54.059540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757bc93-967e-45d9-8552-1386ea748011 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:54.125399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff896887-088d-4d1f-b840-591e3ed781ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:54.126009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff896887-088d-4d1f-b840-591e3ed781ae None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:54.126299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff896887-088d-4d1f-b840-591e3ed781ae None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:54.126579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff896887-088d-4d1f-b840-591e3ed781ae None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:54.134167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff896887-088d-4d1f-b840-591e3ed781ae None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:54.135671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff896887-088d-4d1f-b840-591e3ed781ae None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:54.137641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 85/170] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:53 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:54.138342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff896887-088d-4d1f-b840-591e3ed781ae None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:54.187127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f654f-e81e-4205-864c-a07ff70eaa43 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:54.207144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:54.265909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca695cc-e48b-4e2c-9031-902a38877b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:54.266573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca695cc-e48b-4e2c-9031-902a38877b7e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:54.266864 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca695cc-e48b-4e2c-9031-902a38877b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:54.267147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca695cc-e48b-4e2c-9031-902a38877b7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:54.288215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff896887-088d-4d1f-b840-591e3ed781ae None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:54.293930 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca695cc-e48b-4e2c-9031-902a38877b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:54.297536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-bca695cc-e48b-4e2c-9031-902a38877b7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:54.299160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:54.300027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:08:54.300799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 86/171] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:54.301527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca695cc-e48b-4e2c-9031-902a38877b7e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:54.382503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617474c4-9f6e-43ca-9947-6b65da8f3d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:54.383089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617474c4-9f6e-43ca-9947-6b65da8f3d01 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:54.383385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617474c4-9f6e-43ca-9947-6b65da8f3d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:54.383669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617474c4-9f6e-43ca-9947-6b65da8f3d01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:54.511525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617474c4-9f6e-43ca-9947-6b65da8f3d01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:54.513175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617474c4-9f6e-43ca-9947-6b65da8f3d01 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:54.600372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca695cc-e48b-4e2c-9031-902a38877b7e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:54.684279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7c069-9419-4dcc-884d-20b5bc5f8b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:54.684846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7c069-9419-4dcc-884d-20b5bc5f8b36 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:54.685133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7c069-9419-4dcc-884d-20b5bc5f8b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:54.685443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7c069-9419-4dcc-884d-20b5bc5f8b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:54.686261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7c069-9419-4dcc-884d-20b5bc5f8b36 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3, user.id=b42a77304cdd4ceda1f777fdc9b5ccf9) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:54.686582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7c069-9419-4dcc-884d-20b5bc5f8b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:54.692822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7c069-9419-4dcc-884d-20b5bc5f8b36 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:54.706913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 87/172] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:54 2022] GET /identity/v3/role_assignments?user.id=b42a77304cdd4ceda1f777fdc9b5ccf9&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 608 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:54.707525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7c069-9419-4dcc-884d-20b5bc5f8b36 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:54.737908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:54.753404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 86/173] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:54.755392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685d17-92ee-4382-941f-a1477db1e0c5 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:54.756413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685d17-92ee-4382-941f-a1477db1e0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:54.756966 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685d17-92ee-4382-941f-a1477db1e0c5 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:54.758572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 87/174] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:54.759958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf685d17-92ee-4382-941f-a1477db1e0c5 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:54.766615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:54.920924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7c069-9419-4dcc-884d-20b5bc5f8b36 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:54.995790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be0d05-59ef-4a9d-a8a9-e6dd41a29037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:54.996393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be0d05-59ef-4a9d-a8a9-e6dd41a29037 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:54.996683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be0d05-59ef-4a9d-a8a9-e6dd41a29037 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:54.996966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be0d05-59ef-4a9d-a8a9-e6dd41a29037 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:54.997743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be0d05-59ef-4a9d-a8a9-e6dd41a29037 None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:54.998056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be0d05-59ef-4a9d-a8a9-e6dd41a29037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:54.999470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be0d05-59ef-4a9d-a8a9-e6dd41a29037 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:55.022559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 88/175] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:08:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:55.024261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e82fa-7af8-4bcf-b77c-1653480a9589 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.024565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e82fa-7af8-4bcf-b77c-1653480a9589 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.024855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e82fa-7af8-4bcf-b77c-1653480a9589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.053002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf685d17-92ee-4382-941f-a1477db1e0c5 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:55.058482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-da9e82fa-7af8-4bcf-b77c-1653480a9589 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:55.060017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-da9e82fa-7af8-4bcf-b77c-1653480a9589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:55.094881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0db7cb-30eb-4df6-8387-336d93e12a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:55.095549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0db7cb-30eb-4df6-8387-336d93e12a55 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.095837 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0db7cb-30eb-4df6-8387-336d93e12a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.096117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0db7cb-30eb-4df6-8387-336d93e12a55 None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.097666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0db7cb-30eb-4df6-8387-336d93e12a55 None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:55.105337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0db7cb-30eb-4df6-8387-336d93e12a55 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:55.105337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 88/176] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:54 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:55.105337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0db7cb-30eb-4df6-8387-336d93e12a55 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:55.190415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:55.200461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 89/177] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:55.202056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e82fa-7af8-4bcf-b77c-1653480a9589 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:55.249121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0db7cb-30eb-4df6-8387-336d93e12a55 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:55.258682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:55.303416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3e4ed-04e8-4002-81da-237ab3896e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:55.304898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3e4ed-04e8-4002-81da-237ab3896e8d None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.305651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3e4ed-04e8-4002-81da-237ab3896e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.306384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3e4ed-04e8-4002-81da-237ab3896e8d None admin] PATH_INFO: `/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.308854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3e4ed-04e8-4002-81da-237ab3896e8d None admin] RBAC: Authorizing `identity:get_user(user_id=05a7b57c5cfe4d368e6c200fc07ecf2d)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:55.310808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3e4ed-04e8-4002-81da-237ab3896e8d None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:55.312918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 89/178] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:55 2022] GET /identity/v3/users/05a7b57c5cfe4d368e6c200fc07ecf2d => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:55.314593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a07a1c-0559-4802-a60a-a9d15fae189e None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.315178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a07a1c-0559-4802-a60a-a9d15fae189e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.315638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a07a1c-0559-4802-a60a-a9d15fae189e None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.317126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 90/179] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:55.318162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a07a1c-0559-4802-a60a-a9d15fae189e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:55.373384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e82fa-7af8-4bcf-b77c-1653480a9589 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:55.438308 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f21762-366d-4b83-b948-9f4ef8727297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:55.438950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f21762-366d-4b83-b948-9f4ef8727297 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.439232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f21762-366d-4b83-b948-9f4ef8727297 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.440030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f21762-366d-4b83-b948-9f4ef8727297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.440838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f21762-366d-4b83-b948-9f4ef8727297 None admin] RBAC: Adding query filter params (role.id=c7b4bfc82f3b46cfaa10773dad7cd03a, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3, user.id=05a7b57c5cfe4d368e6c200fc07ecf2d) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:55.441151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f21762-366d-4b83-b948-9f4ef8727297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:55.443255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f21762-366d-4b83-b948-9f4ef8727297 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:55.451900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 90/180] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:55 2022] GET /identity/v3/role_assignments?user.id=05a7b57c5cfe4d368e6c200fc07ecf2d&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=c7b4bfc82f3b46cfaa10773dad7cd03a => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:55.453285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3133-0076-4394-88a1-9335add6d362 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.453621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3133-0076-4394-88a1-9335add6d362 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.454468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3133-0076-4394-88a1-9335add6d362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.477917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-208a3133-0076-4394-88a1-9335add6d362 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:55.478941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a07a1c-0559-4802-a60a-a9d15fae189e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:55.479318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-208a3133-0076-4394-88a1-9335add6d362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:55.562081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 91/181] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:55.562817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a3133-0076-4394-88a1-9335add6d362 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:55.563156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6902d15-6f27-41bd-bdfe-d7b289a52978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:55.563916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902d15-6f27-41bd-bdfe-d7b289a52978 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.564299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902d15-6f27-41bd-bdfe-d7b289a52978 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.564657 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902d15-6f27-41bd-bdfe-d7b289a52978 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.567534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:55.580437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6902d15-6f27-41bd-bdfe-d7b289a52978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-d6902d15-6f27-41bd-bdfe-d7b289a52978 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:55.586684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:55.587643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:55.588422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 91/182] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:08:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:55.594679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746200-e09d-429c-b6b0-caecb8b4c477 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.596995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746200-e09d-429c-b6b0-caecb8b4c477 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.596995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746200-e09d-429c-b6b0-caecb8b4c477 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.596995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 92/183] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:55.598078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4746200-e09d-429c-b6b0-caecb8b4c477 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:55.609202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:55.731034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a3133-0076-4394-88a1-9335add6d362 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:55.755554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4746200-e09d-429c-b6b0-caecb8b4c477 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:55.785105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfd793-defa-4209-886e-2ebb4b2a7b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:55.785803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfd793-defa-4209-886e-2ebb4b2a7b4e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.786140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfd793-defa-4209-886e-2ebb4b2a7b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.786423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfd793-defa-4209-886e-2ebb4b2a7b4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.797448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62517628-e034-44fc-8b0a-ea17465e9568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:55.798184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62517628-e034-44fc-8b0a-ea17465e9568 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.798392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62517628-e034-44fc-8b0a-ea17465e9568 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.798670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62517628-e034-44fc-8b0a-ea17465e9568 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.810042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62517628-e034-44fc-8b0a-ea17465e9568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-62517628-e034-44fc-8b0a-ea17465e9568 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:55.815093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:55.815957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:55.816764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 93/184] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:08:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:55.817442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62517628-e034-44fc-8b0a-ea17465e9568 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:55.888928 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfd793-defa-4209-886e-2ebb4b2a7b4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:55.891582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfd793-defa-4209-886e-2ebb4b2a7b4e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:55.905767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:55.912632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:55.951161 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62517628-e034-44fc-8b0a-ea17465e9568 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:55.984279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd31f-412d-4cf9-92ab-6ed6aa4b2ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:55.984899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd31f-412d-4cf9-92ab-6ed6aa4b2ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.985180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd31f-412d-4cf9-92ab-6ed6aa4b2ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:55.985484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd31f-412d-4cf9-92ab-6ed6aa4b2ef1 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:55.986208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd31f-412d-4cf9-92ab-6ed6aa4b2ef1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:55.986517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd31f-412d-4cf9-92ab-6ed6aa4b2ef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:55.987919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd31f-412d-4cf9-92ab-6ed6aa4b2ef1 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:55.997511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 94/185] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:55.998818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bf501-fdb5-465f-8b8b-294d1534058b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:55.999113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bf501-fdb5-465f-8b8b-294d1534058b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.003800 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 92/186] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:56.003970 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bf501-fdb5-465f-8b8b-294d1534058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.004062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfd793-defa-4209-886e-2ebb4b2a7b4e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:56.018491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-214bf501-fdb5-465f-8b8b-294d1534058b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:56.019682 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-214bf501-fdb5-465f-8b8b-294d1534058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:56.086515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 95/187] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:56.087914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b8add-ead6-4196-83fd-d60e9b4d2648 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.088199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b8add-ead6-4196-83fd-d60e9b4d2648 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.088478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b8add-ead6-4196-83fd-d60e9b4d2648 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.093551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 96/188] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:56.093551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b8add-ead6-4196-83fd-d60e9b4d2648 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:56.108844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfd793-defa-4209-886e-2ebb4b2a7b4e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:56.127295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:56.142692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:56.148261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca471ff3-4072-4a2e-8acc-42acb72b744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:56.148511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca471ff3-4072-4a2e-8acc-42acb72b744e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.148511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca471ff3-4072-4a2e-8acc-42acb72b744e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.148511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca471ff3-4072-4a2e-8acc-42acb72b744e None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.148511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca471ff3-4072-4a2e-8acc-42acb72b744e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:56.148511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca471ff3-4072-4a2e-8acc-42acb72b744e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:56.148511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca471ff3-4072-4a2e-8acc-42acb72b744e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:56.164092 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 93/189] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:08:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:56.164092 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca471ff3-4072-4a2e-8acc-42acb72b744e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:56.204301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b8add-ead6-4196-83fd-d60e9b4d2648 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:56.239047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70eab40-be68-4f54-81d3-b8264cf8424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:56.239730 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eab40-be68-4f54-81d3-b8264cf8424c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.240031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eab40-be68-4f54-81d3-b8264cf8424c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.240348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eab40-be68-4f54-81d3-b8264cf8424c None admin] PATH_INFO: `/v3/users/a194c0358d1f4d2fbec96e59a932bfe3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.253871 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70eab40-be68-4f54-81d3-b8264cf8424c None admin] RBAC: Authorizing `identity:get_user(user_id=a194c0358d1f4d2fbec96e59a932bfe3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:56.255856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70eab40-be68-4f54-81d3-b8264cf8424c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:56.257528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 97/190] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:56 2022] GET /identity/v3/users/a194c0358d1f4d2fbec96e59a932bfe3 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:56.258692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220ed92-f23a-48ea-8abb-ca4e2b60ee50 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.259200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220ed92-f23a-48ea-8abb-ca4e2b60ee50 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.259336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220ed92-f23a-48ea-8abb-ca4e2b60ee50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.271722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca471ff3-4072-4a2e-8acc-42acb72b744e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:56.278293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2220ed92-f23a-48ea-8abb-ca4e2b60ee50 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:56.279479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2220ed92-f23a-48ea-8abb-ca4e2b60ee50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:56.305151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9b31e-60cc-4b5c-93d2-e55f33dc40cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:56.305836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9b31e-60cc-4b5c-93d2-e55f33dc40cc None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.306120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9b31e-60cc-4b5c-93d2-e55f33dc40cc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.306408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9b31e-60cc-4b5c-93d2-e55f33dc40cc None admin] PATH_INFO: `/v3/projects/e6449d9e47674424b968cf98dfa19948` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.307581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9b31e-60cc-4b5c-93d2-e55f33dc40cc None admin] RBAC: Authorizing `identity:get_project(project_id=e6449d9e47674424b968cf98dfa19948)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:56.309035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9b31e-60cc-4b5c-93d2-e55f33dc40cc None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:56.310806 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 94/191] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:56 2022] GET /identity/v3/projects/e6449d9e47674424b968cf98dfa19948 => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:56.311512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9b31e-60cc-4b5c-93d2-e55f33dc40cc None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:56.350471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 98/192] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:56.351246 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220ed92-f23a-48ea-8abb-ca4e2b60ee50 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:56.373257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:56.387363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:56.426535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9b31e-60cc-4b5c-93d2-e55f33dc40cc None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:56.456688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220ed92-f23a-48ea-8abb-ca4e2b60ee50 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:56.461853 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b304051-5cd2-4cf5-ba01-d10259f63624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:56.462585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b304051-5cd2-4cf5-ba01-d10259f63624 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.462873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b304051-5cd2-4cf5-ba01-d10259f63624 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.463151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b304051-5cd2-4cf5-ba01-d10259f63624 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.464305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b304051-5cd2-4cf5-ba01-d10259f63624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:56.465817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b304051-5cd2-4cf5-ba01-d10259f63624 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:56.470060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 95/193] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:56.470060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b304051-5cd2-4cf5-ba01-d10259f63624 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:56.495744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aeb30a-2456-4e3b-bc7d-56ffc8af4712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:56.496391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aeb30a-2456-4e3b-bc7d-56ffc8af4712 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.496673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aeb30a-2456-4e3b-bc7d-56ffc8af4712 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.496955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aeb30a-2456-4e3b-bc7d-56ffc8af4712 None admin] PATH_INFO: `/v3/users/a194c0358d1f4d2fbec96e59a932bfe3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.498272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aeb30a-2456-4e3b-bc7d-56ffc8af4712 None admin] RBAC: Authorizing `identity:get_user(user_id=a194c0358d1f4d2fbec96e59a932bfe3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:56.504169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aeb30a-2456-4e3b-bc7d-56ffc8af4712 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:56.504169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 99/194] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:56 2022] GET /identity/v3/users/a194c0358d1f4d2fbec96e59a932bfe3 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:56.504169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aeb30a-2456-4e3b-bc7d-56ffc8af4712 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:56.586972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b304051-5cd2-4cf5-ba01-d10259f63624 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:56.604155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:56.619209 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aeb30a-2456-4e3b-bc7d-56ffc8af4712 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:56.620759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:56.622519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b1e70-6474-42e7-9040-69f27cce4634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:56.623149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b1e70-6474-42e7-9040-69f27cce4634 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.623435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b1e70-6474-42e7-9040-69f27cce4634 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.623723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b1e70-6474-42e7-9040-69f27cce4634 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.657106 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a76227-12a9-4d61-b4d1-b2d04531dfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:56.657760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a76227-12a9-4d61-b4d1-b2d04531dfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.658049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a76227-12a9-4d61-b4d1-b2d04531dfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.659074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a76227-12a9-4d61-b4d1-b2d04531dfc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.659165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a76227-12a9-4d61-b4d1-b2d04531dfc0 None admin] RBAC: Adding query filter params (role.id=c4fc73849d664b2588bd2de7372aa839, scope.project.id=e6449d9e47674424b968cf98dfa19948, user.id=05a7b57c5cfe4d368e6c200fc07ecf2d) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:56.659431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a76227-12a9-4d61-b4d1-b2d04531dfc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:56.660889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a76227-12a9-4d61-b4d1-b2d04531dfc0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:56.667431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 100/195] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:56 2022] GET /identity/v3/role_assignments?user.id=05a7b57c5cfe4d368e6c200fc07ecf2d&scope.project.id=e6449d9e47674424b968cf98dfa19948&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:56.668239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a76227-12a9-4d61-b4d1-b2d04531dfc0 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:56.710318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b1e70-6474-42e7-9040-69f27cce4634 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:56.711824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b1e70-6474-42e7-9040-69f27cce4634 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:56.787200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a76227-12a9-4d61-b4d1-b2d04531dfc0 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:56.827004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 96/196] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:56.827701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b1e70-6474-42e7-9040-69f27cce4634 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:56.830558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4eb62-12ac-4494-afc9-2048adadbdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:56.831220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4eb62-12ac-4494-afc9-2048adadbdbf None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.831765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4eb62-12ac-4494-afc9-2048adadbdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.832056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4eb62-12ac-4494-afc9-2048adadbdbf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.847247 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4eb62-12ac-4494-afc9-2048adadbdbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:56.849581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-0cd4eb62-12ac-4494-afc9-2048adadbdbf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:56.852307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:56.853373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:56.854244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 101/197] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:56.854923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4eb62-12ac-4494-afc9-2048adadbdbf None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:56.889456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:56.953047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b1e70-6474-42e7-9040-69f27cce4634 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:56.965323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4eb62-12ac-4494-afc9-2048adadbdbf None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:56.987392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955955b-ed47-4ac8-8db6-947375b5e4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:56.988016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955955b-ed47-4ac8-8db6-947375b5e4f5 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:56.988303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955955b-ed47-4ac8-8db6-947375b5e4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:56.988581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955955b-ed47-4ac8-8db6-947375b5e4f5 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:56.989272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955955b-ed47-4ac8-8db6-947375b5e4f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:56.991598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955955b-ed47-4ac8-8db6-947375b5e4f5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:56.999955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15e3cc-c374-41e9-b139-83e0b66319c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:57.000584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15e3cc-c374-41e9-b139-83e0b66319c4 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:57.000964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15e3cc-c374-41e9-b139-83e0b66319c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:57.001155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15e3cc-c374-41e9-b139-83e0b66319c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:57.065596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 97/198] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:08:56 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:08:57.066475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955955b-ed47-4ac8-8db6-947375b5e4f5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:57.076237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15e3cc-c374-41e9-b139-83e0b66319c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:57.078432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15e3cc-c374-41e9-b139-83e0b66319c4 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:57.111043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:57.154210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:57.195952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955955b-ed47-4ac8-8db6-947375b5e4f5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:57.200892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 102/199] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:57.201696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15e3cc-c374-41e9-b139-83e0b66319c4 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:57.247145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cc632-e757-4d7a-81d4-e453b86c161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:57.247845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cc632-e757-4d7a-81d4-e453b86c161e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:57.248134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cc632-e757-4d7a-81d4-e453b86c161e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:57.248414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cc632-e757-4d7a-81d4-e453b86c161e None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:57.249656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cc632-e757-4d7a-81d4-e453b86c161e None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:57.251142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cc632-e757-4d7a-81d4-e453b86c161e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:57.252825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 98/200] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:57 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:57.253711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cc632-e757-4d7a-81d4-e453b86c161e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:57.322601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15e3cc-c374-41e9-b139-83e0b66319c4 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:57.363543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:57.377396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fc779-fa07-48a0-8da8-d0cd2b73acd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:57.377970 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc779-fa07-48a0-8da8-d0cd2b73acd3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:57.378265 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc779-fa07-48a0-8da8-d0cd2b73acd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:57.383921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc779-fa07-48a0-8da8-d0cd2b73acd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:57.383921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fc779-fa07-48a0-8da8-d0cd2b73acd3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:57.383921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fc779-fa07-48a0-8da8-d0cd2b73acd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:57.383921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fc779-fa07-48a0-8da8-d0cd2b73acd3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:57.392082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 103/201] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:08:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:57.393514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fc779-fa07-48a0-8da8-d0cd2b73acd3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:57.395579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cc632-e757-4d7a-81d4-e453b86c161e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:57.443900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05f0ea-248a-4117-bc2a-26fa618f584e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:57.444938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05f0ea-248a-4117-bc2a-26fa618f584e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:57.446119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05f0ea-248a-4117-bc2a-26fa618f584e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:57.446635 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05f0ea-248a-4117-bc2a-26fa618f584e None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:57.448580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05f0ea-248a-4117-bc2a-26fa618f584e None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:57.451256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05f0ea-248a-4117-bc2a-26fa618f584e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:57.454410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 99/202] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:57 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:57.455511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05f0ea-248a-4117-bc2a-26fa618f584e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:57.468871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:57.539295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fc779-fa07-48a0-8da8-d0cd2b73acd3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:57.577682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05f0ea-248a-4117-bc2a-26fa618f584e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:57.578855 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9f0db-b325-4fc0-b6c5-604d4ad9470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:57.579488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9f0db-b325-4fc0-b6c5-604d4ad9470a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:57.579774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9f0db-b325-4fc0-b6c5-604d4ad9470a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:57.580053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9f0db-b325-4fc0-b6c5-604d4ad9470a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:57.580862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9f0db-b325-4fc0-b6c5-604d4ad9470a None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=0660a31d921d44188bb944fcdbb9b027, user.id=a194c0358d1f4d2fbec96e59a932bfe3) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:57.581175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9f0db-b325-4fc0-b6c5-604d4ad9470a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:57.582702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9f0db-b325-4fc0-b6c5-604d4ad9470a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:57.589376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 104/203] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:57 2022] GET /identity/v3/role_assignments?user.id=a194c0358d1f4d2fbec96e59a932bfe3&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:57.590191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9f0db-b325-4fc0-b6c5-604d4ad9470a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:57.650527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebf18f-081e-43b5-9404-20bc32d5888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:57.651124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebf18f-081e-43b5-9404-20bc32d5888e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:57.652029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebf18f-081e-43b5-9404-20bc32d5888e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:57.652317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebf18f-081e-43b5-9404-20bc32d5888e None admin] PATH_INFO: `/v3/users/fb8360e10f594311b18d2d4276f01ee9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:57.655963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebf18f-081e-43b5-9404-20bc32d5888e None admin] RBAC: Authorizing `identity:get_user(user_id=fb8360e10f594311b18d2d4276f01ee9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:57.657876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebf18f-081e-43b5-9404-20bc32d5888e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:57.663602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 100/204] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:57 2022] GET /identity/v3/users/fb8360e10f594311b18d2d4276f01ee9 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:57.672052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebf18f-081e-43b5-9404-20bc32d5888e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:57.713341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:57.794457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9f0db-b325-4fc0-b6c5-604d4ad9470a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:57.811224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:57.836195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebf18f-081e-43b5-9404-20bc32d5888e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:57.842823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5b71d-56b6-4400-8e60-3878526ca80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:57.843523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5b71d-56b6-4400-8e60-3878526ca80d None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:57.843810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5b71d-56b6-4400-8e60-3878526ca80d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:57.844088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5b71d-56b6-4400-8e60-3878526ca80d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:57.844952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da5b71d-56b6-4400-8e60-3878526ca80d None admin] RBAC: Adding query filter params (role.id=c7b4bfc82f3b46cfaa10773dad7cd03a, scope.project.id=0660a31d921d44188bb944fcdbb9b027, user.id=a194c0358d1f4d2fbec96e59a932bfe3) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:57.845272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da5b71d-56b6-4400-8e60-3878526ca80d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:57.846771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da5b71d-56b6-4400-8e60-3878526ca80d None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:57.854971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 105/205] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:57 2022] GET /identity/v3/role_assignments?user.id=a194c0358d1f4d2fbec96e59a932bfe3&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=c7b4bfc82f3b46cfaa10773dad7cd03a => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:57.885718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c2f53-4686-4e22-821c-289258d9073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:57.886375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c2f53-4686-4e22-821c-289258d9073d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:57.886660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c2f53-4686-4e22-821c-289258d9073d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:57.886937 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c2f53-4686-4e22-821c-289258d9073d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:57.962882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c2f53-4686-4e22-821c-289258d9073d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:57.964339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c2f53-4686-4e22-821c-289258d9073d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:58.048759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:58.073029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 101/206] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:08:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:08:58.078574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5b71d-56b6-4400-8e60-3878526ca80d None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:58.205440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5b71d-56b6-4400-8e60-3878526ca80d None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:58.233796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:58.242997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9358c-a6ae-4d6c-a53a-b3eb9785489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:58.243649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9358c-a6ae-4d6c-a53a-b3eb9785489c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:58.243959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9358c-a6ae-4d6c-a53a-b3eb9785489c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:58.253093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9358c-a6ae-4d6c-a53a-b3eb9785489c None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:58.253093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9358c-a6ae-4d6c-a53a-b3eb9785489c None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:58.253093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9358c-a6ae-4d6c-a53a-b3eb9785489c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:58.253093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 106/207] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:08:58 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:58.259064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c2f53-4686-4e22-821c-289258d9073d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:58.409520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c2f53-4686-4e22-821c-289258d9073d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:58.454844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456448e8-2889-4af6-930a-7caf07c4b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:58.455483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456448e8-2889-4af6-930a-7caf07c4b21f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:58.455773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456448e8-2889-4af6-930a-7caf07c4b21f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:58.458804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456448e8-2889-4af6-930a-7caf07c4b21f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:58.458804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456448e8-2889-4af6-930a-7caf07c4b21f None admin] RBAC: Adding query filter params (role.id=8e94dcc6eec7482fab6c8aba670a4b6c, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3, user.id=fb8360e10f594311b18d2d4276f01ee9) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:58.458804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456448e8-2889-4af6-930a-7caf07c4b21f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:58.458804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456448e8-2889-4af6-930a-7caf07c4b21f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:58.465428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 102/208] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:08:58 2022] GET /identity/v3/role_assignments?user.id=fb8360e10f594311b18d2d4276f01ee9&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=8e94dcc6eec7482fab6c8aba670a4b6c => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:08:59.371178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c06e2a-7dcd-4b6c-a6cc-edbb2f07d95b None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:59.371643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c06e2a-7dcd-4b6c-a6cc-edbb2f07d95b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:59.371726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c06e2a-7dcd-4b6c-a6cc-edbb2f07d95b None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:59.373084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 107/209] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:59.378669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203ff94-8de7-49ea-8a87-c6c5a7c881bd None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:59.378967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203ff94-8de7-49ea-8a87-c6c5a7c881bd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:59.379249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203ff94-8de7-49ea-8a87-c6c5a7c881bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:59.400977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9203ff94-8de7-49ea-8a87-c6c5a7c881bd None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:59.402228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9203ff94-8de7-49ea-8a87-c6c5a7c881bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:59.438705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:59.473559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 103/210] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:08:59.496151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c06e2a-7dcd-4b6c-a6cc-edbb2f07d95b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:59.560497 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24158b-66f0-4f6e-83af-633110ea6308 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:59.561120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24158b-66f0-4f6e-83af-633110ea6308 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:59.561653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24158b-66f0-4f6e-83af-633110ea6308 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:59.563511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 104/211] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:08:59.569246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba883-1733-4c08-a1b8-c0e26bf587ff None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:59.569829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba883-1733-4c08-a1b8-c0e26bf587ff None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:59.570314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba883-1733-4c08-a1b8-c0e26bf587ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:59.591596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2caba883-1733-4c08-a1b8-c0e26bf587ff None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:08:59.593199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2caba883-1733-4c08-a1b8-c0e26bf587ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:08:59.606772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c06e2a-7dcd-4b6c-a6cc-edbb2f07d95b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:59.643466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadafa3-5ee0-4331-83c3-87e144c1abc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:59.644033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadafa3-5ee0-4331-83c3-87e144c1abc1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:59.644318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadafa3-5ee0-4331-83c3-87e144c1abc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:59.644597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadafa3-5ee0-4331-83c3-87e144c1abc1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:59.659193 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadafa3-5ee0-4331-83c3-87e144c1abc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-deadafa3-5ee0-4331-83c3-87e144c1abc1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:08:59.666022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:08:59.666931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:08:59.667710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:08:59.670337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 108/212] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:08:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:08:59.676579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadafa3-5ee0-4331-83c3-87e144c1abc1 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:59.678919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 105/213] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:08:59.706187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caba883-1733-4c08-a1b8-c0e26bf587ff None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:59.732385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:59.794222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadafa3-5ee0-4331-83c3-87e144c1abc1 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:59.821664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:59.825059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caba883-1733-4c08-a1b8-c0e26bf587ff None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:59.833234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27d4a5-de80-4c14-9e1c-04ff79d4ea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:59.834304 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27d4a5-de80-4c14-9e1c-04ff79d4ea36 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:59.834588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27d4a5-de80-4c14-9e1c-04ff79d4ea36 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:59.834870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27d4a5-de80-4c14-9e1c-04ff79d4ea36 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:59.835612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27d4a5-de80-4c14-9e1c-04ff79d4ea36 None admin] RBAC: Adding query filter params (name=member) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:08:59.835923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27d4a5-de80-4c14-9e1c-04ff79d4ea36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:59.837441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27d4a5-de80-4c14-9e1c-04ff79d4ea36 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:59.846659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 109/214] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:08:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:59.853294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27d4a5-de80-4c14-9e1c-04ff79d4ea36 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:59.871094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b48f-7ebb-4d1f-ae24-45e26e3bbf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:08:59.872130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b48f-7ebb-4d1f-ae24-45e26e3bbf32 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:08:59.872629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b48f-7ebb-4d1f-ae24-45e26e3bbf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:08:59.873097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b48f-7ebb-4d1f-ae24-45e26e3bbf32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:08:59.874781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893b48f-7ebb-4d1f-ae24-45e26e3bbf32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:08:59.876500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893b48f-7ebb-4d1f-ae24-45e26e3bbf32 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:08:59.878671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 106/215] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:08:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:08:59.885075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b48f-7ebb-4d1f-ae24-45e26e3bbf32 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:08:59.972948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27d4a5-de80-4c14-9e1c-04ff79d4ea36 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:08:59.981485 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:08:59.998647 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b48f-7ebb-4d1f-ae24-45e26e3bbf32 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:00.018968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674634c-b6e2-4e4f-aa3b-24105cc2e3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:00.020254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674634c-b6e2-4e4f-aa3b-24105cc2e3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.020771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674634c-b6e2-4e4f-aa3b-24105cc2e3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.021696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674634c-b6e2-4e4f-aa3b-24105cc2e3e6 None admin] PATH_INFO: `/v3/users/3eafdec75d7d4f43b584a052a0dd3b9f` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.039085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674634c-b6e2-4e4f-aa3b-24105cc2e3e6 None admin] RBAC: Authorizing `identity:get_user(user_id=3eafdec75d7d4f43b584a052a0dd3b9f)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:00.040115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de588df6-8628-4a43-a59b-41ab1f7fcb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:00.040620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674634c-b6e2-4e4f-aa3b-24105cc2e3e6 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:00.040798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de588df6-8628-4a43-a59b-41ab1f7fcb36 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.041053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de588df6-8628-4a43-a59b-41ab1f7fcb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.041360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de588df6-8628-4a43-a59b-41ab1f7fcb36 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.042593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 110/216] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:08:59 2022] GET /identity/v3/users/3eafdec75d7d4f43b584a052a0dd3b9f => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:00.042698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de588df6-8628-4a43-a59b-41ab1f7fcb36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:00.043841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de588df6-8628-4a43-a59b-41ab1f7fcb36 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:00.048919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0bf87-d451-416b-9ea9-4d545e33e89e None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.049207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0bf87-d451-416b-9ea9-4d545e33e89e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.049517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0bf87-d451-416b-9ea9-4d545e33e89e None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.052914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 111/217] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:00.052914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0bf87-d451-416b-9ea9-4d545e33e89e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:00.141481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 107/218] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:08:59 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:09:00.150918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041566ac-2796-4902-94a0-0d05efc621fa None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.151212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041566ac-2796-4902-94a0-0d05efc621fa None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.151523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041566ac-2796-4902-94a0-0d05efc621fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.162064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0bf87-d451-416b-9ea9-4d545e33e89e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:00.203055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f8c00-7110-43f9-ad31-4cee664c848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:00.203696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f8c00-7110-43f9-ad31-4cee664c848f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.203980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f8c00-7110-43f9-ad31-4cee664c848f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.204257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f8c00-7110-43f9-ad31-4cee664c848f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.206422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-041566ac-2796-4902-94a0-0d05efc621fa None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:00.207650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-041566ac-2796-4902-94a0-0d05efc621fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:00.235227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:00.251671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:00.308086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f8c00-7110-43f9-ad31-4cee664c848f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:00.310063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f8c00-7110-43f9-ad31-4cee664c848f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:00.319170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 108/219] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:00.344774 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041566ac-2796-4902-94a0-0d05efc621fa None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:00.433493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 112/220] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:00.435361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391be92-d08b-4f09-9158-ba520224b03f None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.435663 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391be92-d08b-4f09-9158-ba520224b03f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.435943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391be92-d08b-4f09-9158-ba520224b03f None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.437252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 113/221] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:00.439063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391be92-d08b-4f09-9158-ba520224b03f None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:00.480713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041566ac-2796-4902-94a0-0d05efc621fa None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:00.557683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:00.568400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce07032-2814-43d2-a9bd-a4642018a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:00.569070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce07032-2814-43d2-a9bd-a4642018a812 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.569372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce07032-2814-43d2-a9bd-a4642018a812 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.569653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce07032-2814-43d2-a9bd-a4642018a812 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.584499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce07032-2814-43d2-a9bd-a4642018a812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-cce07032-2814-43d2-a9bd-a4642018a812 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:00.590511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:00.591414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:00.592198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:00.592805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 109/222] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:09:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:00.595246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c874e6-4cb3-4f7e-8c93-9e20f24ad89d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.595503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c874e6-4cb3-4f7e-8c93-9e20f24ad89d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.595886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c874e6-4cb3-4f7e-8c93-9e20f24ad89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.606992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391be92-d08b-4f09-9158-ba520224b03f None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:00.617901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e6c874e6-4cb3-4f7e-8c93-9e20f24ad89d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:00.619439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e6c874e6-4cb3-4f7e-8c93-9e20f24ad89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:00.633151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:00.649878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34afd4f-3401-4869-ae2b-b9f6524826c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:00.650887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34afd4f-3401-4869-ae2b-b9f6524826c2 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.651451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34afd4f-3401-4869-ae2b-b9f6524826c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.651902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34afd4f-3401-4869-ae2b-b9f6524826c2 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.653922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34afd4f-3401-4869-ae2b-b9f6524826c2 None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:00.656306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34afd4f-3401-4869-ae2b-b9f6524826c2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:00.659672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 114/223] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:00 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:00.660627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34afd4f-3401-4869-ae2b-b9f6524826c2 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:00.719574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 110/224] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:00.721117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c874e6-4cb3-4f7e-8c93-9e20f24ad89d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:00.801943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34afd4f-3401-4869-ae2b-b9f6524826c2 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:00.836179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:00.841598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f6e2a-bcee-475d-81e3-52dec10d043c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:00.842626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6e2a-bcee-475d-81e3-52dec10d043c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.843115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6e2a-bcee-475d-81e3-52dec10d043c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.843592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6e2a-bcee-475d-81e3-52dec10d043c None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.844527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f6e2a-bcee-475d-81e3-52dec10d043c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:00.845068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f6e2a-bcee-475d-81e3-52dec10d043c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:00.846894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f6e2a-bcee-475d-81e3-52dec10d043c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:00.862526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 115/225] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:00.863763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f6e2a-bcee-475d-81e3-52dec10d043c None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:00.873085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c874e6-4cb3-4f7e-8c93-9e20f24ad89d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:00.918653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f1559-4eae-4014-8bbe-3b7ee2935419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:00.919821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1559-4eae-4014-8bbe-3b7ee2935419 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:00.920307 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1559-4eae-4014-8bbe-3b7ee2935419 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:00.920814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1559-4eae-4014-8bbe-3b7ee2935419 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:00.921868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f1559-4eae-4014-8bbe-3b7ee2935419 None admin] RBAC: Adding query filter params (role.id=c4fc73849d664b2588bd2de7372aa839, scope.project.id=0660a31d921d44188bb944fcdbb9b027, user.id=3eafdec75d7d4f43b584a052a0dd3b9f) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:00.922481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f1559-4eae-4014-8bbe-3b7ee2935419 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:00.924220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f1559-4eae-4014-8bbe-3b7ee2935419 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:00.931740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 111/226] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:00 2022] GET /identity/v3/role_assignments?user.id=3eafdec75d7d4f43b584a052a0dd3b9f&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:00.932887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f1559-4eae-4014-8bbe-3b7ee2935419 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:00.959861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:01.036206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f6e2a-bcee-475d-81e3-52dec10d043c None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:01.112552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2ab2f-4924-47d2-a91f-a35aca857a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:01.113148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2ab2f-4924-47d2-a91f-a35aca857a5d None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:01.113492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2ab2f-4924-47d2-a91f-a35aca857a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:01.113777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2ab2f-4924-47d2-a91f-a35aca857a5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:01.125860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2ab2f-4924-47d2-a91f-a35aca857a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-eaa2ab2f-4924-47d2-a91f-a35aca857a5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:01.132159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:01.133042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:01.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:01.136289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f1559-4eae-4014-8bbe-3b7ee2935419 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:01.137415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 116/227] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:09:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:01.140271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2ab2f-4924-47d2-a91f-a35aca857a5d None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:01.175376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece2e33-2d60-42dd-b762-581ed01799ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:01.176448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece2e33-2d60-42dd-b762-581ed01799ee None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:01.176932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece2e33-2d60-42dd-b762-581ed01799ee None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:01.178131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece2e33-2d60-42dd-b762-581ed01799ee None admin] PATH_INFO: `/v3/users/a194c0358d1f4d2fbec96e59a932bfe3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:01.179704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece2e33-2d60-42dd-b762-581ed01799ee None admin] RBAC: Authorizing `identity:get_user(user_id=a194c0358d1f4d2fbec96e59a932bfe3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:01.181505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece2e33-2d60-42dd-b762-581ed01799ee None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:01.183359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 112/228] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:00 2022] GET /identity/v3/users/a194c0358d1f4d2fbec96e59a932bfe3 => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:01.189114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece2e33-2d60-42dd-b762-581ed01799ee None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:01.205754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:01.265254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2ab2f-4924-47d2-a91f-a35aca857a5d None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:01.317827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:01.325664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece2e33-2d60-42dd-b762-581ed01799ee None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:01.342024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bac142-1f4c-46ec-8f46-69d15d6ffd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:01.343086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bac142-1f4c-46ec-8f46-69d15d6ffd26 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:01.343645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bac142-1f4c-46ec-8f46-69d15d6ffd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:01.344243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bac142-1f4c-46ec-8f46-69d15d6ffd26 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:01.345210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bac142-1f4c-46ec-8f46-69d15d6ffd26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:01.351159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bac142-1f4c-46ec-8f46-69d15d6ffd26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:01.354883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bac142-1f4c-46ec-8f46-69d15d6ffd26 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:01.381995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 117/229] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:01.389335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bac142-1f4c-46ec-8f46-69d15d6ffd26 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:01.411221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b19cb-496d-4b44-856b-0343c24d1e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:01.413143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b19cb-496d-4b44-856b-0343c24d1e5a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:01.413677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b19cb-496d-4b44-856b-0343c24d1e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:01.414213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b19cb-496d-4b44-856b-0343c24d1e5a None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:01.415812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b19cb-496d-4b44-856b-0343c24d1e5a None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:01.417663 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b19cb-496d-4b44-856b-0343c24d1e5a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:01.426126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 113/230] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:01 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:01.431380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b19cb-496d-4b44-856b-0343c24d1e5a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:01.534596 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bac142-1f4c-46ec-8f46-69d15d6ffd26 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:01.559554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b19cb-496d-4b44-856b-0343c24d1e5a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:01.573885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:01.608096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c61223-f626-42b8-bbbd-69a5a5360230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:01.609984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61223-f626-42b8-bbbd-69a5a5360230 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:01.610475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61223-f626-42b8-bbbd-69a5a5360230 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:01.610959 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61223-f626-42b8-bbbd-69a5a5360230 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027/users/a194c0358d1f4d2fbec96e59a932bfe3/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:01.613201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c61223-f626-42b8-bbbd-69a5a5360230 None admin] RBAC: Authorizing `identity:create_grant(project_id=0660a31d921d44188bb944fcdbb9b027, user_id=a194c0358d1f4d2fbec96e59a932bfe3, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:01.615061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c61223-f626-42b8-bbbd-69a5a5360230 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:01.631080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f82a9d-6f98-40ab-a7ed-d597ce671f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:01.632119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f82a9d-6f98-40ab-a7ed-d597ce671f8d None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:01.632631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f82a9d-6f98-40ab-a7ed-d597ce671f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:01.633106 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f82a9d-6f98-40ab-a7ed-d597ce671f8d None admin] PATH_INFO: `/v3/users/a194c0358d1f4d2fbec96e59a932bfe3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:01.634692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f82a9d-6f98-40ab-a7ed-d597ce671f8d None admin] RBAC: Authorizing `identity:get_user(user_id=a194c0358d1f4d2fbec96e59a932bfe3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:01.636592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f82a9d-6f98-40ab-a7ed-d597ce671f8d None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:01.638909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 118/231] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:01 2022] GET /identity/v3/users/a194c0358d1f4d2fbec96e59a932bfe3 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:01.644408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f82a9d-6f98-40ab-a7ed-d597ce671f8d None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:01.683709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 114/232] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:09:01 2022] PUT /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027/users/a194c0358d1f4d2fbec96e59a932bfe3/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:09:01.777441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f82a9d-6f98-40ab-a7ed-d597ce671f8d None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:01.838383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7ec3b-dc87-4dcd-b931-44eb3ab318e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:01.839074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ec3b-dc87-4dcd-b931-44eb3ab318e1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:01.839410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ec3b-dc87-4dcd-b931-44eb3ab318e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:01.839698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ec3b-dc87-4dcd-b931-44eb3ab318e1 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:01.840981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7ec3b-dc87-4dcd-b931-44eb3ab318e1 None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:01.842592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7ec3b-dc87-4dcd-b931-44eb3ab318e1 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:01.844580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 119/233] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:01 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:01.851008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c61223-f626-42b8-bbbd-69a5a5360230 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:01.855174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:01.985324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c61223-f626-42b8-bbbd-69a5a5360230 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:02.023074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdfd72-7c07-45a6-b937-de9b68e30a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:02.023820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdfd72-7c07-45a6-b937-de9b68e30a81 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:02.024112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdfd72-7c07-45a6-b937-de9b68e30a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:02.024400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdfd72-7c07-45a6-b937-de9b68e30a81 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027/users/a194c0358d1f4d2fbec96e59a932bfe3/roles/c7b4bfc82f3b46cfaa10773dad7cd03a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:02.026813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdfd72-7c07-45a6-b937-de9b68e30a81 None admin] RBAC: Authorizing `identity:create_grant(project_id=0660a31d921d44188bb944fcdbb9b027, user_id=a194c0358d1f4d2fbec96e59a932bfe3, role_id=c7b4bfc82f3b46cfaa10773dad7cd03a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:02.028305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdfd72-7c07-45a6-b937-de9b68e30a81 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:02.155557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 115/234] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:09:01 2022] PUT /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027/users/a194c0358d1f4d2fbec96e59a932bfe3/roles/c7b4bfc82f3b46cfaa10773dad7cd03a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:02.426589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c29bb-011c-4d58-a322-89d9c953dd8c None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:02.427412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c29bb-011c-4d58-a322-89d9c953dd8c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:02.427988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c29bb-011c-4d58-a322-89d9c953dd8c None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:02.429661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 120/235] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:02.435273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d532f-7ff2-4768-98b3-ed20785f3bc9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:02.435817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d532f-7ff2-4768-98b3-ed20785f3bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:02.436299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d532f-7ff2-4768-98b3-ed20785f3bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:02.460080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5d6d532f-7ff2-4768-98b3-ed20785f3bc9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:02.461282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5d6d532f-7ff2-4768-98b3-ed20785f3bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:02.579065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:02.590926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 116/236] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:02.615054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c29bb-011c-4d58-a322-89d9c953dd8c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:02.688831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:02.690991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a470a4-8a96-4d64-a5d8-3e0f2216a24a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:02.691284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a470a4-8a96-4d64-a5d8-3e0f2216a24a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:02.691566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a470a4-8a96-4d64-a5d8-3e0f2216a24a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:02.694049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 117/237] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:02.702670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990c3f2-8ed4-40bc-8d46-13c4c9925c2f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:02.702957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990c3f2-8ed4-40bc-8d46-13c4c9925c2f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:02.703235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990c3f2-8ed4-40bc-8d46-13c4c9925c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:02.742356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3990c3f2-8ed4-40bc-8d46-13c4c9925c2f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:02.743557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3990c3f2-8ed4-40bc-8d46-13c4c9925c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:02.747592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c29bb-011c-4d58-a322-89d9c953dd8c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:02.818708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f53b4a-0a11-4311-9b7e-278dc3ae60f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:02.819359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f53b4a-0a11-4311-9b7e-278dc3ae60f8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:02.819642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f53b4a-0a11-4311-9b7e-278dc3ae60f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:02.819918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f53b4a-0a11-4311-9b7e-278dc3ae60f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:02.840887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f53b4a-0a11-4311-9b7e-278dc3ae60f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-29f53b4a-0a11-4311-9b7e-278dc3ae60f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:02.846580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:02.847664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:02.848583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:02.855998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 121/238] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:02.858619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f53b4a-0a11-4311-9b7e-278dc3ae60f8 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:02.877411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 118/239] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:02.924459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990c3f2-8ed4-40bc-8d46-13c4c9925c2f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:03.026787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f53b4a-0a11-4311-9b7e-278dc3ae60f8 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:03.050543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:03.091929 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e890b-8390-4b55-a611-2d3b7ece5e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:03.092576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e890b-8390-4b55-a611-2d3b7ece5e8f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:03.092859 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e890b-8390-4b55-a611-2d3b7ece5e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:03.093132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e890b-8390-4b55-a611-2d3b7ece5e8f None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:03.093903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e890b-8390-4b55-a611-2d3b7ece5e8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:03.094212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e890b-8390-4b55-a611-2d3b7ece5e8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:03.095647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e890b-8390-4b55-a611-2d3b7ece5e8f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:03.110980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 122/240] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:03.119249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e890b-8390-4b55-a611-2d3b7ece5e8f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:03.125299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990c3f2-8ed4-40bc-8d46-13c4c9925c2f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:03.185283 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:03.197836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e4d2a-73aa-4616-a8fb-2e0c7f5f33c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:03.198507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e4d2a-73aa-4616-a8fb-2e0c7f5f33c5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:03.198786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e4d2a-73aa-4616-a8fb-2e0c7f5f33c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:03.199061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e4d2a-73aa-4616-a8fb-2e0c7f5f33c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:03.200241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e4d2a-73aa-4616-a8fb-2e0c7f5f33c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:03.201736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e4d2a-73aa-4616-a8fb-2e0c7f5f33c5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:03.203341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 119/241] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:03.209521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e4d2a-73aa-4616-a8fb-2e0c7f5f33c5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:03.315067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e890b-8390-4b55-a611-2d3b7ece5e8f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:03.342700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e4d2a-73aa-4616-a8fb-2e0c7f5f33c5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:03.351881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc640fa-b10c-4543-8dea-b71cfb05eecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:03.352594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc640fa-b10c-4543-8dea-b71cfb05eecb None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:03.352872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc640fa-b10c-4543-8dea-b71cfb05eecb None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:03.353147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc640fa-b10c-4543-8dea-b71cfb05eecb None admin] PATH_INFO: `/v3/users/261d0a584b394fd8add4e99cab7c07a3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:03.380809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa8517-46a9-4c70-a7b8-bb066bbf3610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:03.381667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa8517-46a9-4c70-a7b8-bb066bbf3610 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:03.381958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa8517-46a9-4c70-a7b8-bb066bbf3610 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:03.382240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa8517-46a9-4c70-a7b8-bb066bbf3610 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:03.382944 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa8517-46a9-4c70-a7b8-bb066bbf3610 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:03.384414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa8517-46a9-4c70-a7b8-bb066bbf3610 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:03.404342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc640fa-b10c-4543-8dea-b71cfb05eecb None admin] RBAC: Authorizing `identity:get_user(user_id=261d0a584b394fd8add4e99cab7c07a3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:03.405891 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc640fa-b10c-4543-8dea-b71cfb05eecb None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:03.407910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 123/242] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:03 2022] GET /identity/v3/users/261d0a584b394fd8add4e99cab7c07a3 => generated 297 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:03.413885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc640fa-b10c-4543-8dea-b71cfb05eecb None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:03.446375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:03.494934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 120/243] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:09:03 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:09:03.527268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc640fa-b10c-4543-8dea-b71cfb05eecb None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:03.566609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce0eeb-96cf-44e3-b2b8-fe3ae70afd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:03.567320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce0eeb-96cf-44e3-b2b8-fe3ae70afd23 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:03.568229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce0eeb-96cf-44e3-b2b8-fe3ae70afd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:03.568523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce0eeb-96cf-44e3-b2b8-fe3ae70afd23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:03.579413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf460825-e2e9-4c02-9668-76b70832bcf7 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:03.579934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf460825-e2e9-4c02-9668-76b70832bcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:03.580007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf460825-e2e9-4c02-9668-76b70832bcf7 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:03.581396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 121/244] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:09:03.587737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffb979-27aa-4890-9ff2-2682851753d0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:03.588025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffb979-27aa-4890-9ff2-2682851753d0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:03.588299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffb979-27aa-4890-9ff2-2682851753d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:03.622452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d7ffb979-27aa-4890-9ff2-2682851753d0 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:03.623724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d7ffb979-27aa-4890-9ff2-2682851753d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:03.714615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:03.718120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce0eeb-96cf-44e3-b2b8-fe3ae70afd23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:03.719690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce0eeb-96cf-44e3-b2b8-fe3ae70afd23 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:03.743456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 122/245] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:03.774250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:03.774691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffb979-27aa-4890-9ff2-2682851753d0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:03.881786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 124/246] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:09:03.888796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce0eeb-96cf-44e3-b2b8-fe3ae70afd23 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:03.930250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffb979-27aa-4890-9ff2-2682851753d0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:03.976376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4a4f1-959c-4914-9b1a-adc26b498f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:03.977000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4a4f1-959c-4914-9b1a-adc26b498f51 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:03.977283 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4a4f1-959c-4914-9b1a-adc26b498f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:03.977591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4a4f1-959c-4914-9b1a-adc26b498f51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:03.991547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4a4f1-959c-4914-9b1a-adc26b498f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8cb4a4f1-959c-4914-9b1a-adc26b498f51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:03.996604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:03.997576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:03.998371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:03.999503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 123/247] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:04.008151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce0eeb-96cf-44e3-b2b8-fe3ae70afd23 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:04.010203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4a4f1-959c-4914-9b1a-adc26b498f51 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:04.047295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35d2c-c0c3-4a7b-9d36-7e758024ead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:04.048073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35d2c-c0c3-4a7b-9d36-7e758024ead7 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:04.048412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35d2c-c0c3-4a7b-9d36-7e758024ead7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:04.048928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35d2c-c0c3-4a7b-9d36-7e758024ead7 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:04.051273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b35d2c-c0c3-4a7b-9d36-7e758024ead7 None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:04.055101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b35d2c-c0c3-4a7b-9d36-7e758024ead7 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:04.057453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 125/248] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:03 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:04.057920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:04.064325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35d2c-c0c3-4a7b-9d36-7e758024ead7 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:04.065461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:04.138432 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4a4f1-959c-4914-9b1a-adc26b498f51 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:04.178151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c672d-1f80-4ead-8545-648a553fc82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:04.178906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c672d-1f80-4ead-8545-648a553fc82f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:04.179095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c672d-1f80-4ead-8545-648a553fc82f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:04.179379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c672d-1f80-4ead-8545-648a553fc82f None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:04.180186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c672d-1f80-4ead-8545-648a553fc82f None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:04.180419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c672d-1f80-4ead-8545-648a553fc82f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:04.181867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c672d-1f80-4ead-8545-648a553fc82f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:04.189171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35d2c-c0c3-4a7b-9d36-7e758024ead7 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:04.193196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 124/249] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:04.199582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c672d-1f80-4ead-8545-648a553fc82f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:04.230108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54616a9b-dde3-483c-8037-e46bf66883f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:04.231093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616a9b-dde3-483c-8037-e46bf66883f8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:04.231564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616a9b-dde3-483c-8037-e46bf66883f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:04.232032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616a9b-dde3-483c-8037-e46bf66883f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:04.233131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54616a9b-dde3-483c-8037-e46bf66883f8 None admin] RBAC: Adding query filter params (role.id=8e94dcc6eec7482fab6c8aba670a4b6c, scope.project.id=0660a31d921d44188bb944fcdbb9b027, user.id=261d0a584b394fd8add4e99cab7c07a3) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:04.233651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54616a9b-dde3-483c-8037-e46bf66883f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:04.235475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54616a9b-dde3-483c-8037-e46bf66883f8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:04.243704 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 126/250] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:04 2022] GET /identity/v3/role_assignments?user.id=261d0a584b394fd8add4e99cab7c07a3&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=8e94dcc6eec7482fab6c8aba670a4b6c => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:04.245590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd4640-df89-497d-a0ca-00d07c7244b4 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:04.246067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd4640-df89-497d-a0ca-00d07c7244b4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:04.246530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd4640-df89-497d-a0ca-00d07c7244b4 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:04.248457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 127/251] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:04.256101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58c5e0-f5fe-4a8f-bfd6-2a00697adcc4 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:04.256617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58c5e0-f5fe-4a8f-bfd6-2a00697adcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:04.256912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58c5e0-f5fe-4a8f-bfd6-2a00697adcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:04.278020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-1c58c5e0-f5fe-4a8f-bfd6-2a00697adcc4 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:04.279254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1c58c5e0-f5fe-4a8f-bfd6-2a00697adcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:04.342907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c672d-1f80-4ead-8545-648a553fc82f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:04.356801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:04.369746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:04.371606 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 128/252] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:04.395885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58c5e0-f5fe-4a8f-bfd6-2a00697adcc4 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:04.397614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef48dd-54df-4c90-91a1-743f87c0de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:04.398301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef48dd-54df-4c90-91a1-743f87c0de8e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:04.398590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef48dd-54df-4c90-91a1-743f87c0de8e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:04.398932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef48dd-54df-4c90-91a1-743f87c0de8e None admin] PATH_INFO: `/v3/users/3eafdec75d7d4f43b584a052a0dd3b9f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:04.400823 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef48dd-54df-4c90-91a1-743f87c0de8e None admin] RBAC: Authorizing `identity:get_user(user_id=3eafdec75d7d4f43b584a052a0dd3b9f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:04.401950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef48dd-54df-4c90-91a1-743f87c0de8e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:04.408861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 125/253] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:04 2022] GET /identity/v3/users/3eafdec75d7d4f43b584a052a0dd3b9f => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:04.409971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef48dd-54df-4c90-91a1-743f87c0de8e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:04.506474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58c5e0-f5fe-4a8f-bfd6-2a00697adcc4 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:04.545298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76640e-1df5-445c-8664-8ac768696f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:04.546459 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76640e-1df5-445c-8664-8ac768696f86 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:04.546743 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76640e-1df5-445c-8664-8ac768696f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:04.547095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76640e-1df5-445c-8664-8ac768696f86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:04.564421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76640e-1df5-445c-8664-8ac768696f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-0b76640e-1df5-445c-8664-8ac768696f86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:04.570745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:04.571630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:04.572464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 129/254] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:09:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:04.578036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76640e-1df5-445c-8664-8ac768696f86 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:04.617370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef48dd-54df-4c90-91a1-743f87c0de8e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:04.667098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:04.694368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccb0d0-e0a4-4ca5-9baa-5810b969467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:04.695103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccb0d0-e0a4-4ca5-9baa-5810b969467d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:04.695403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccb0d0-e0a4-4ca5-9baa-5810b969467d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:04.695674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccb0d0-e0a4-4ca5-9baa-5810b969467d None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:04.697931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccb0d0-e0a4-4ca5-9baa-5810b969467d None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:04.699409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccb0d0-e0a4-4ca5-9baa-5810b969467d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:04.701268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 126/255] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:04 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:04.708310 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccb0d0-e0a4-4ca5-9baa-5810b969467d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:04.736939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76640e-1df5-445c-8664-8ac768696f86 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:04.790423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:04.825733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bff99-8cf1-4154-9ead-f590f7b06809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:04.826378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bff99-8cf1-4154-9ead-f590f7b06809 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:04.826665 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bff99-8cf1-4154-9ead-f590f7b06809 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:04.826944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bff99-8cf1-4154-9ead-f590f7b06809 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:04.827687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bff99-8cf1-4154-9ead-f590f7b06809 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:04.827998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bff99-8cf1-4154-9ead-f590f7b06809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:04.829423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bff99-8cf1-4154-9ead-f590f7b06809 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:04.843494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 130/256] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:04.846436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c08bba-fd16-421a-8d2a-233f2c8a2f21 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:04.846727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c08bba-fd16-421a-8d2a-233f2c8a2f21 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:04.847009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c08bba-fd16-421a-8d2a-233f2c8a2f21 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:04.848572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 131/257] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:04.854542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c08bba-fd16-421a-8d2a-233f2c8a2f21 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:04.940362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccb0d0-e0a4-4ca5-9baa-5810b969467d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:05.002532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4f95c-7972-4116-ab6e-56fc6859a3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:05.003711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4f95c-7972-4116-ab6e-56fc6859a3e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:05.004275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4f95c-7972-4116-ab6e-56fc6859a3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:05.004761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4f95c-7972-4116-ab6e-56fc6859a3e4 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027/users/3eafdec75d7d4f43b584a052a0dd3b9f/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:05.007726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4f95c-7972-4116-ab6e-56fc6859a3e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0660a31d921d44188bb944fcdbb9b027, user_id=3eafdec75d7d4f43b584a052a0dd3b9f, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:05.009509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4f95c-7972-4116-ab6e-56fc6859a3e4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:05.027023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c08bba-fd16-421a-8d2a-233f2c8a2f21 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:05.044919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:05.064017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acc599-cb16-45e4-b9d7-8e02b2597bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:05.064793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acc599-cb16-45e4-b9d7-8e02b2597bea None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:05.065109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acc599-cb16-45e4-b9d7-8e02b2597bea None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:05.067649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acc599-cb16-45e4-b9d7-8e02b2597bea None admin] PATH_INFO: `/v3/users/a194c0358d1f4d2fbec96e59a932bfe3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:05.067649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acc599-cb16-45e4-b9d7-8e02b2597bea None admin] RBAC: Authorizing `identity:get_user(user_id=a194c0358d1f4d2fbec96e59a932bfe3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:05.068609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acc599-cb16-45e4-b9d7-8e02b2597bea None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:05.070728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 132/258] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:04 2022] GET /identity/v3/users/a194c0358d1f4d2fbec96e59a932bfe3 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:05.072086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bc513-f41f-4f1b-a6c6-8d8bb8b64324 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:05.072384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bc513-f41f-4f1b-a6c6-8d8bb8b64324 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:05.072740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bc513-f41f-4f1b-a6c6-8d8bb8b64324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:05.092204 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 127/259] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:09:04 2022] PUT /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027/users/3eafdec75d7d4f43b584a052a0dd3b9f/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:05.093898 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4f95c-7972-4116-ab6e-56fc6859a3e4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:05.094308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b25bc513-f41f-4f1b-a6c6-8d8bb8b64324 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:05.097520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b25bc513-f41f-4f1b-a6c6-8d8bb8b64324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:05.174704 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 133/260] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:05.200439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bc513-f41f-4f1b-a6c6-8d8bb8b64324 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:05.280907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:05.296181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4f95c-7972-4116-ab6e-56fc6859a3e4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:05.393100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bc513-f41f-4f1b-a6c6-8d8bb8b64324 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:05.394056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08663221-066b-406f-822b-8c15b53b5ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:05.394714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08663221-066b-406f-822b-8c15b53b5ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:05.395002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08663221-066b-406f-822b-8c15b53b5ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:05.395279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08663221-066b-406f-822b-8c15b53b5ad0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:05.429718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:05.445280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd84a-11e4-49ff-bf29-0ad3247fba95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:05.445976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd84a-11e4-49ff-bf29-0ad3247fba95 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:05.446263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd84a-11e4-49ff-bf29-0ad3247fba95 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:05.450791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd84a-11e4-49ff-bf29-0ad3247fba95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:05.470493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abd84a-11e4-49ff-bf29-0ad3247fba95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-a4abd84a-11e4-49ff-bf29-0ad3247fba95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:05.484212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:05.485146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:05.486292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:05.490296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 134/261] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:09:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:05.494762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd84a-11e4-49ff-bf29-0ad3247fba95 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:05.557822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08663221-066b-406f-822b-8c15b53b5ad0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:05.559244 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08663221-066b-406f-822b-8c15b53b5ad0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:05.717177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:05.727103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 128/262] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:05.727643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd84a-11e4-49ff-bf29-0ad3247fba95 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:05.732187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08663221-066b-406f-822b-8c15b53b5ad0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:05.798273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec2b95-f6cf-44dd-a84a-8663676e65ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:05.798920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2b95-f6cf-44dd-a84a-8663676e65ef None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:05.799335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2b95-f6cf-44dd-a84a-8663676e65ef None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:05.799698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2b95-f6cf-44dd-a84a-8663676e65ef None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:05.800599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec2b95-f6cf-44dd-a84a-8663676e65ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:05.800971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec2b95-f6cf-44dd-a84a-8663676e65ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:05.803343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec2b95-f6cf-44dd-a84a-8663676e65ef None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:05.828013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 135/263] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:05.846017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec2b95-f6cf-44dd-a84a-8663676e65ef None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:05.863351 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08663221-066b-406f-822b-8c15b53b5ad0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:05.905579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e04b56-e678-48ab-957a-1d2a41489052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:05.906820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e04b56-e678-48ab-957a-1d2a41489052 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:05.907294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e04b56-e678-48ab-957a-1d2a41489052 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:05.908154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e04b56-e678-48ab-957a-1d2a41489052 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:05.909821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e04b56-e678-48ab-957a-1d2a41489052 None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:05.912235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e04b56-e678-48ab-957a-1d2a41489052 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:05.919540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 129/264] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:05 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:05.926333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e04b56-e678-48ab-957a-1d2a41489052 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:05.928879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:06.033691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:06.083568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec2b95-f6cf-44dd-a84a-8663676e65ef None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:06.100446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e04b56-e678-48ab-957a-1d2a41489052 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:06.170597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8f951-14dc-4fff-a35a-7d5757b9e8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:06.171538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8f951-14dc-4fff-a35a-7d5757b9e8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:06.172016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8f951-14dc-4fff-a35a-7d5757b9e8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:06.172498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8f951-14dc-4fff-a35a-7d5757b9e8e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:06.173508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8f951-14dc-4fff-a35a-7d5757b9e8e0 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=0660a31d921d44188bb944fcdbb9b027, user.id=a194c0358d1f4d2fbec96e59a932bfe3) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:06.174005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8f951-14dc-4fff-a35a-7d5757b9e8e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:06.175173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1e3af-243b-4608-8bda-3338283c5b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:06.175844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e3af-243b-4608-8bda-3338283c5b20 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:06.176136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e3af-243b-4608-8bda-3338283c5b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:06.176353 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8f951-14dc-4fff-a35a-7d5757b9e8e0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:06.176452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e3af-243b-4608-8bda-3338283c5b20 None admin] PATH_INFO: `/v3/users/a194c0358d1f4d2fbec96e59a932bfe3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:06.183208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1e3af-243b-4608-8bda-3338283c5b20 None admin] RBAC: Authorizing `identity:get_user(user_id=a194c0358d1f4d2fbec96e59a932bfe3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:06.183208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1e3af-243b-4608-8bda-3338283c5b20 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:06.183585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 136/265] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:05 2022] GET /identity/v3/users/a194c0358d1f4d2fbec96e59a932bfe3 => generated 283 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:06.192092 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 130/266] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:05 2022] GET /identity/v3/role_assignments?user.id=a194c0358d1f4d2fbec96e59a932bfe3&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:06.194253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1e3af-243b-4608-8bda-3338283c5b20 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:06.343725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1e3af-243b-4608-8bda-3338283c5b20 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:06.350726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:06.402101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00977a-f273-499a-a27b-3b9597376708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:06.402762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00977a-f273-499a-a27b-3b9597376708 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:06.403048 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00977a-f273-499a-a27b-3b9597376708 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:06.403328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00977a-f273-499a-a27b-3b9597376708 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:06.493511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00977a-f273-499a-a27b-3b9597376708 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:06.495114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00977a-f273-499a-a27b-3b9597376708 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:06.602311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 137/267] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:06.607994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8f951-14dc-4fff-a35a-7d5757b9e8e0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:06.729733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8f951-14dc-4fff-a35a-7d5757b9e8e0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:06.772261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:06.780309 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ff14f-c9e2-4d88-aa38-a6a5d8c24bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:06.780979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ff14f-c9e2-4d88-aa38-a6a5d8c24bad None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:06.781272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ff14f-c9e2-4d88-aa38-a6a5d8c24bad None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:06.781604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ff14f-c9e2-4d88-aa38-a6a5d8c24bad None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:06.782813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ff14f-c9e2-4d88-aa38-a6a5d8c24bad None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:06.784293 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ff14f-c9e2-4d88-aa38-a6a5d8c24bad None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:06.786650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 131/268] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:06 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:06.792630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00977a-f273-499a-a27b-3b9597376708 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:06.795885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:06.922473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00977a-f273-499a-a27b-3b9597376708 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:06.958912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910d31b-5b40-481f-95d1-e77b64639f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:06.959545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910d31b-5b40-481f-95d1-e77b64639f60 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:06.959837 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910d31b-5b40-481f-95d1-e77b64639f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:06.960121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910d31b-5b40-481f-95d1-e77b64639f60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:06.960931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910d31b-5b40-481f-95d1-e77b64639f60 None admin] RBAC: Adding query filter params (role.id=c7b4bfc82f3b46cfaa10773dad7cd03a, scope.project.id=0660a31d921d44188bb944fcdbb9b027, user.id=a194c0358d1f4d2fbec96e59a932bfe3) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:06.961248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910d31b-5b40-481f-95d1-e77b64639f60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:06.962750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910d31b-5b40-481f-95d1-e77b64639f60 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:06.968969 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 138/269] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:06 2022] GET /identity/v3/role_assignments?user.id=a194c0358d1f4d2fbec96e59a932bfe3&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=c7b4bfc82f3b46cfaa10773dad7cd03a => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:06.976937 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123faf9d-5f6f-4a36-a240-c656dc9598d2 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:06.977597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123faf9d-5f6f-4a36-a240-c656dc9598d2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:06.978107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123faf9d-5f6f-4a36-a240-c656dc9598d2 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:06.980042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 132/270] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:06.980239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c4f78-5dfe-492b-92ef-4aeb7c30a6cb None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:06.982470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c4f78-5dfe-492b-92ef-4aeb7c30a6cb None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:06.982470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c4f78-5dfe-492b-92ef-4aeb7c30a6cb None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:06.982470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 139/271] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:09:06.986795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314f8e9-d074-47d1-91ff-b12c2336968c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:06.987355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314f8e9-d074-47d1-91ff-b12c2336968c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:06.987457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e4c53-10b0-43d5-ac1e-7280cd2a05cb None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:06.987699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e4c53-10b0-43d5-ac1e-7280cd2a05cb None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:06.987985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e4c53-10b0-43d5-ac1e-7280cd2a05cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:06.988334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314f8e9-d074-47d1-91ff-b12c2336968c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:07.009158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2314f8e9-d074-47d1-91ff-b12c2336968c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:07.010736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2314f8e9-d074-47d1-91ff-b12c2336968c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:07.018526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-343e4c53-10b0-43d5-ac1e-7280cd2a05cb None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:07.019774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-343e4c53-10b0-43d5-ac1e-7280cd2a05cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:07.089788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:07.094825 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 140/272] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:07.124462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 133/273] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:07.135961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e4c53-10b0-43d5-ac1e-7280cd2a05cb None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:07.173023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314f8e9-d074-47d1-91ff-b12c2336968c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:07.294248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e4c53-10b0-43d5-ac1e-7280cd2a05cb None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:07.327260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314f8e9-d074-47d1-91ff-b12c2336968c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:07.340241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774d144-485f-40be-9cb1-4d54ace81b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:07.341098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774d144-485f-40be-9cb1-4d54ace81b35 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:07.341231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774d144-485f-40be-9cb1-4d54ace81b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:07.341541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774d144-485f-40be-9cb1-4d54ace81b35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:07.342703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774d144-485f-40be-9cb1-4d54ace81b35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:07.344182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774d144-485f-40be-9cb1-4d54ace81b35 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:07.348610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 141/274] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:07.348610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a121dff-055f-4976-8d2b-21bfefe16e7b None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:07.348610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a121dff-055f-4976-8d2b-21bfefe16e7b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:07.348610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a121dff-055f-4976-8d2b-21bfefe16e7b None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:07.349078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 142/275] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:07.358744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f43b1-dc91-4dab-808c-18de1e9d089d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:07.359114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f43b1-dc91-4dab-808c-18de1e9d089d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:07.359473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f43b1-dc91-4dab-808c-18de1e9d089d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:07.374046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:07.381192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-a39f43b1-dc91-4dab-808c-18de1e9d089d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:07.382437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a39f43b1-dc91-4dab-808c-18de1e9d089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:07.384503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f126ae3-e992-466a-aa03-a8faefc8a6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:07.385225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f126ae3-e992-466a-aa03-a8faefc8a6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:07.385630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f126ae3-e992-466a-aa03-a8faefc8a6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:07.385979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f126ae3-e992-466a-aa03-a8faefc8a6f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:07.398208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f126ae3-e992-466a-aa03-a8faefc8a6f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7f126ae3-e992-466a-aa03-a8faefc8a6f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:07.403280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:07.404187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:07.404975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 134/276] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:07.405686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f126ae3-e992-466a-aa03-a8faefc8a6f0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:07.453336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:07.461276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 143/277] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:07.462192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f43b1-dc91-4dab-808c-18de1e9d089d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:07.530491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f126ae3-e992-466a-aa03-a8faefc8a6f0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:07.568369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d47c4-54a8-4e01-acd1-c3d9375d9efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:07.569373 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d47c4-54a8-4e01-acd1-c3d9375d9efa None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:07.569877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d47c4-54a8-4e01-acd1-c3d9375d9efa None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:07.570337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d47c4-54a8-4e01-acd1-c3d9375d9efa None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:07.571191 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d47c4-54a8-4e01-acd1-c3d9375d9efa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:07.571436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f43b1-dc91-4dab-808c-18de1e9d089d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:07.573144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d47c4-54a8-4e01-acd1-c3d9375d9efa None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:07.607832 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319f2dc-36f0-415e-b0a0-eadd38558779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:07.608461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319f2dc-36f0-415e-b0a0-eadd38558779 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:07.608746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319f2dc-36f0-415e-b0a0-eadd38558779 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:07.609025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319f2dc-36f0-415e-b0a0-eadd38558779 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:07.609791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319f2dc-36f0-415e-b0a0-eadd38558779 None admin] RBAC: Adding query filter params (name=member) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:07.610100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319f2dc-36f0-415e-b0a0-eadd38558779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:07.611486 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319f2dc-36f0-415e-b0a0-eadd38558779 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:07.620670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 144/278] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:07.621527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319f2dc-36f0-415e-b0a0-eadd38558779 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:07.685012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:07.695633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 135/279] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:09:07 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:09:07.706697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d47c4-54a8-4e01-acd1-c3d9375d9efa None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:07.738684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319f2dc-36f0-415e-b0a0-eadd38558779 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:07.780855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:07.793552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760390e4-71a8-4923-a35c-30642c05a2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:07.794992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760390e4-71a8-4923-a35c-30642c05a2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:07.794992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760390e4-71a8-4923-a35c-30642c05a2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:07.795328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760390e4-71a8-4923-a35c-30642c05a2f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:07.815051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760390e4-71a8-4923-a35c-30642c05a2f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-760390e4-71a8-4923-a35c-30642c05a2f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:07.825879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:07.827001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:07.827786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:07.829780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 145/280] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:07.833922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760390e4-71a8-4923-a35c-30642c05a2f1 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:07.871923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d47c4-54a8-4e01-acd1-c3d9375d9efa None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:07.920000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d77b3d-a01f-40ba-a1e8-6a1141caea81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:07.920000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d77b3d-a01f-40ba-a1e8-6a1141caea81 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:07.920000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d77b3d-a01f-40ba-a1e8-6a1141caea81 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:07.920000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d77b3d-a01f-40ba-a1e8-6a1141caea81 None admin] PATH_INFO: `/v3/users/f3afc34a0afc4194980849cf16767354` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:07.944434 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d77b3d-a01f-40ba-a1e8-6a1141caea81 None admin] RBAC: Authorizing `identity:get_user(user_id=f3afc34a0afc4194980849cf16767354)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:07.945974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d77b3d-a01f-40ba-a1e8-6a1141caea81 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:07.947916 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 136/281] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:07 2022] GET /identity/v3/users/f3afc34a0afc4194980849cf16767354 => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:07.953929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d77b3d-a01f-40ba-a1e8-6a1141caea81 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:07.976559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760390e4-71a8-4923-a35c-30642c05a2f1 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:08.007358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:08.030752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001308cd-23fa-4507-abe6-50e2bc1113ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:08.031760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001308cd-23fa-4507-abe6-50e2bc1113ea None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:08.047666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001308cd-23fa-4507-abe6-50e2bc1113ea None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:08.047666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001308cd-23fa-4507-abe6-50e2bc1113ea None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:08.047666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001308cd-23fa-4507-abe6-50e2bc1113ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:08.047666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001308cd-23fa-4507-abe6-50e2bc1113ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:08.047666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001308cd-23fa-4507-abe6-50e2bc1113ea None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:08.047666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 146/282] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:08.051528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001308cd-23fa-4507-abe6-50e2bc1113ea None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:08.087257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:08.091197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d77b3d-a01f-40ba-a1e8-6a1141caea81 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:08.138273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1034b3-786e-4cfd-9deb-acadcc3afe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:08.138924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1034b3-786e-4cfd-9deb-acadcc3afe60 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:08.139212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1034b3-786e-4cfd-9deb-acadcc3afe60 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:08.140285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1034b3-786e-4cfd-9deb-acadcc3afe60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:08.141117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1034b3-786e-4cfd-9deb-acadcc3afe60 None admin] RBAC: Adding query filter params (role.id=c4fc73849d664b2588bd2de7372aa839, user.id=f3afc34a0afc4194980849cf16767354, scope.system=all) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:08.141469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1034b3-786e-4cfd-9deb-acadcc3afe60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:08.143463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1034b3-786e-4cfd-9deb-acadcc3afe60 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:08.150625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 137/283] 10.222.0.69 () {56 vars in 1308 bytes} [Thu Jul 28 06:09:07 2022] GET /identity/v3/role_assignments?user.id=f3afc34a0afc4194980849cf16767354&scope.system=all&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:08.181556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001308cd-23fa-4507-abe6-50e2bc1113ea None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:08.221692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65c06a-cad8-4af1-b21c-52d6a4d67b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:08.222618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65c06a-cad8-4af1-b21c-52d6a4d67b95 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:08.223079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65c06a-cad8-4af1-b21c-52d6a4d67b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:08.223564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65c06a-cad8-4af1-b21c-52d6a4d67b95 None admin] PATH_INFO: `/v3/users/261d0a584b394fd8add4e99cab7c07a3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:08.225615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65c06a-cad8-4af1-b21c-52d6a4d67b95 None admin] RBAC: Authorizing `identity:get_user(user_id=261d0a584b394fd8add4e99cab7c07a3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:08.227495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65c06a-cad8-4af1-b21c-52d6a4d67b95 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:08.229751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 147/284] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:08 2022] GET /identity/v3/users/261d0a584b394fd8add4e99cab7c07a3 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:08.238742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1034b3-786e-4cfd-9deb-acadcc3afe60 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:08.241172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e60fcc-8a97-4c18-af01-b7bbd8143d2d None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:08.241817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e60fcc-8a97-4c18-af01-b7bbd8143d2d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:08.242250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e60fcc-8a97-4c18-af01-b7bbd8143d2d None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:08.243822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 148/285] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:09:08.254386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700f4de-46a0-4c67-a388-6a990076d22e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:08.255519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700f4de-46a0-4c67-a388-6a990076d22e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:08.255962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700f4de-46a0-4c67-a388-6a990076d22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:08.278493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f700f4de-46a0-4c67-a388-6a990076d22e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:08.279827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f700f4de-46a0-4c67-a388-6a990076d22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:08.334904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:08.367413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 149/286] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:08.390858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700f4de-46a0-4c67-a388-6a990076d22e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:08.407766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1034b3-786e-4cfd-9deb-acadcc3afe60 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:08.458691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bc625-42b1-4817-bd31-5446a704932a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:08.459361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc625-42b1-4817-bd31-5446a704932a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:08.459650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc625-42b1-4817-bd31-5446a704932a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:08.459930 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc625-42b1-4817-bd31-5446a704932a None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:08.466070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bc625-42b1-4817-bd31-5446a704932a None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:08.467648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bc625-42b1-4817-bd31-5446a704932a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:08.469285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 138/287] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:08 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:08.475858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bc625-42b1-4817-bd31-5446a704932a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:08.514702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:08.519843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700f4de-46a0-4c67-a388-6a990076d22e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:08.577259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35178a-87b1-47e2-977a-7ab0a9e86a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:08.578342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35178a-87b1-47e2-977a-7ab0a9e86a05 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:08.578830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35178a-87b1-47e2-977a-7ab0a9e86a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:08.579328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35178a-87b1-47e2-977a-7ab0a9e86a05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:08.592366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35178a-87b1-47e2-977a-7ab0a9e86a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-8a35178a-87b1-47e2-977a-7ab0a9e86a05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:08.597827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:08.598963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:08.600019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:08.602903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 150/288] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:08.610469 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35178a-87b1-47e2-977a-7ab0a9e86a05 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:08.623240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bc625-42b1-4817-bd31-5446a704932a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:08.677505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1006c-4a12-4408-aff7-74de081efec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:08.678194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1006c-4a12-4408-aff7-74de081efec8 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:08.678479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1006c-4a12-4408-aff7-74de081efec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:08.678756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1006c-4a12-4408-aff7-74de081efec8 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027/users/261d0a584b394fd8add4e99cab7c07a3/roles/8e94dcc6eec7482fab6c8aba670a4b6c` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:08.681575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1006c-4a12-4408-aff7-74de081efec8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0660a31d921d44188bb944fcdbb9b027, user_id=261d0a584b394fd8add4e99cab7c07a3, role_id=8e94dcc6eec7482fab6c8aba670a4b6c)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:08.683054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1006c-4a12-4408-aff7-74de081efec8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:08.699575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:08.772211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35178a-87b1-47e2-977a-7ab0a9e86a05 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:08.810342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb94b24-895d-45cb-8bf8-70445d22173e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:08.810968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94b24-895d-45cb-8bf8-70445d22173e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:08.811252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94b24-895d-45cb-8bf8-70445d22173e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:08.811524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94b24-895d-45cb-8bf8-70445d22173e None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:08.812250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb94b24-895d-45cb-8bf8-70445d22173e None admin] RBAC: Adding query filter params (name=member) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:08.812558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb94b24-895d-45cb-8bf8-70445d22173e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:08.813974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb94b24-895d-45cb-8bf8-70445d22173e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:08.823253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 151/289] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:08.830342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb94b24-895d-45cb-8bf8-70445d22173e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:08.838401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 139/290] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:09:08 2022] PUT /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027/users/261d0a584b394fd8add4e99cab7c07a3/roles/8e94dcc6eec7482fab6c8aba670a4b6c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:09:09.026838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:09.031235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb94b24-895d-45cb-8bf8-70445d22173e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:09.069976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63575e19-e257-43f2-bbb8-18172a595988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:09.070639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63575e19-e257-43f2-bbb8-18172a595988 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:09.070922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63575e19-e257-43f2-bbb8-18172a595988 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:09.071200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63575e19-e257-43f2-bbb8-18172a595988 None admin] PATH_INFO: `/v3/users/3eafdec75d7d4f43b584a052a0dd3b9f` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:09.072486 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63575e19-e257-43f2-bbb8-18172a595988 None admin] RBAC: Authorizing `identity:get_user(user_id=3eafdec75d7d4f43b584a052a0dd3b9f)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:09.074021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63575e19-e257-43f2-bbb8-18172a595988 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:09.075829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 152/291] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:08 2022] GET /identity/v3/users/3eafdec75d7d4f43b584a052a0dd3b9f => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:09.086531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1006c-4a12-4408-aff7-74de081efec8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:09.193565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:09.206780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1006c-4a12-4408-aff7-74de081efec8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:09.259441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab876-2761-4a40-bc02-9b747075b10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:09.260487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab876-2761-4a40-bc02-9b747075b10c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:09.260995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab876-2761-4a40-bc02-9b747075b10c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:09.261502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab876-2761-4a40-bc02-9b747075b10c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:09.369459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ab876-2761-4a40-bc02-9b747075b10c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:09.370947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ab876-2761-4a40-bc02-9b747075b10c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:09.470003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:09.491643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 140/292] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:09.497648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63575e19-e257-43f2-bbb8-18172a595988 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:09.669821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63575e19-e257-43f2-bbb8-18172a595988 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:09.717009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341a1fe-3aae-436e-9075-a189d6f9f5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:09.718063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341a1fe-3aae-436e-9075-a189d6f9f5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:09.718546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341a1fe-3aae-436e-9075-a189d6f9f5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:09.719053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341a1fe-3aae-436e-9075-a189d6f9f5e0 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:09.720415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341a1fe-3aae-436e-9075-a189d6f9f5e0 None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:09.722096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341a1fe-3aae-436e-9075-a189d6f9f5e0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:09.724630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 153/293] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:09 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:09.730641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab876-2761-4a40-bc02-9b747075b10c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:09.910517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab876-2761-4a40-bc02-9b747075b10c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:09.953664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67608689-ddf8-4576-a03d-2f09227e2fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:09.954214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67608689-ddf8-4576-a03d-2f09227e2fec None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:09.954504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67608689-ddf8-4576-a03d-2f09227e2fec None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:09.954789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67608689-ddf8-4576-a03d-2f09227e2fec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:09.955590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67608689-ddf8-4576-a03d-2f09227e2fec None admin] RBAC: Adding query filter params (role.id=c4fc73849d664b2588bd2de7372aa839, scope.project.id=0660a31d921d44188bb944fcdbb9b027, user.id=3eafdec75d7d4f43b584a052a0dd3b9f) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:09.955907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67608689-ddf8-4576-a03d-2f09227e2fec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:09.957377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67608689-ddf8-4576-a03d-2f09227e2fec None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:09.968875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 141/294] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:09 2022] GET /identity/v3/role_assignments?user.id=3eafdec75d7d4f43b584a052a0dd3b9f&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:10.278658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e607e-daf8-44d4-a0a1-30d32374d835 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:10.279665 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e607e-daf8-44d4-a0a1-30d32374d835 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:10.280152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e607e-daf8-44d4-a0a1-30d32374d835 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:10.282045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 154/295] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:10.287296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347dd60-c82d-41e1-bbc4-04f84179c5e2 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:10.287588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347dd60-c82d-41e1-bbc4-04f84179c5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:10.287874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347dd60-c82d-41e1-bbc4-04f84179c5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:10.314053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7347dd60-c82d-41e1-bbc4-04f84179c5e2 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:10.315265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7347dd60-c82d-41e1-bbc4-04f84179c5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:10.385414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:10.407020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 142/296] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:10.432661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e607e-daf8-44d4-a0a1-30d32374d835 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:10.478165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:10.553082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e607e-daf8-44d4-a0a1-30d32374d835 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:10.592447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42063b2d-9cbd-4562-820d-5e43a01bd98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:10.593111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42063b2d-9cbd-4562-820d-5e43a01bd98d None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:10.593467 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42063b2d-9cbd-4562-820d-5e43a01bd98d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:10.593753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42063b2d-9cbd-4562-820d-5e43a01bd98d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:10.594926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42063b2d-9cbd-4562-820d-5e43a01bd98d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:10.596388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42063b2d-9cbd-4562-820d-5e43a01bd98d None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:10.598043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 155/297] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:10.604694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347dd60-c82d-41e1-bbc4-04f84179c5e2 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:10.649346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0bf84-ecdf-4eff-8753-3d495856a2ed None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:10.649992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0bf84-ecdf-4eff-8753-3d495856a2ed None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:10.650479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0bf84-ecdf-4eff-8753-3d495856a2ed None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:10.652335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 156/298] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:10.657330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49385f0-23f3-4288-a44a-a4a03f2d337b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:10.657728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49385f0-23f3-4288-a44a-a4a03f2d337b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:10.658015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49385f0-23f3-4288-a44a-a4a03f2d337b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:10.677741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b49385f0-23f3-4288-a44a-a4a03f2d337b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:10.678931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b49385f0-23f3-4288-a44a-a4a03f2d337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:10.715129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347dd60-c82d-41e1-bbc4-04f84179c5e2 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:10.752302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71884c70-b695-4e46-aee7-843e49c595ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:10.752979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71884c70-b695-4e46-aee7-843e49c595ce None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:10.753407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 157/299] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:10.755279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032f497-8890-4187-b68b-3d999e7c8b9f None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:10.755623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032f497-8890-4187-b68b-3d999e7c8b9f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:10.755847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032f497-8890-4187-b68b-3d999e7c8b9f None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:10.757250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 158/300] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:10.757505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71884c70-b695-4e46-aee7-843e49c595ce None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:10.757806 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71884c70-b695-4e46-aee7-843e49c595ce None admin] PATH_INFO: `/v3/groups` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:10.758501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71884c70-b695-4e46-aee7-843e49c595ce None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:10.759975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71884c70-b695-4e46-aee7-843e49c595ce None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:10.762519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ad387-5a4b-4e30-a3f1-702c315d88c2 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:10.762816 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ad387-5a4b-4e30-a3f1-702c315d88c2 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:10.763102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ad387-5a4b-4e30-a3f1-702c315d88c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:10.783335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-9f1ad387-5a4b-4e30-a3f1-702c315d88c2 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:10.784565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9f1ad387-5a4b-4e30-a3f1-702c315d88c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:10.812331 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:10.837549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 143/301] 10.222.0.69 () {60 vars in 1134 bytes} [Thu Jul 28 06:09:10 2022] POST /identity/v3/groups => generated 229 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:09:10.838321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71884c70-b695-4e46-aee7-843e49c595ce None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:10.870683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 159/302] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:10.894377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ad387-5a4b-4e30-a3f1-702c315d88c2 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:10.903363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:10.973441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71884c70-b695-4e46-aee7-843e49c595ce None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:11.009584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ad387-5a4b-4e30-a3f1-702c315d88c2 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:11.016623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f83d50-dc65-4af4-8970-f9f588ae1e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:11.017290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f83d50-dc65-4af4-8970-f9f588ae1e12 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.017618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f83d50-dc65-4af4-8970-f9f588ae1e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.017903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f83d50-dc65-4af4-8970-f9f588ae1e12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.031193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f83d50-dc65-4af4-8970-f9f588ae1e12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-59f83d50-dc65-4af4-8970-f9f588ae1e12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:11.036660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:11.037817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:11.038790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 144/303] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:11.045186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f83d50-dc65-4af4-8970-f9f588ae1e12 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:11.060953 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298750bf-885c-4f8f-9a25-9a969cbcc58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:11.061664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298750bf-885c-4f8f-9a25-9a969cbcc58c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.061957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298750bf-885c-4f8f-9a25-9a969cbcc58c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.062238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298750bf-885c-4f8f-9a25-9a969cbcc58c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.078853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298750bf-885c-4f8f-9a25-9a969cbcc58c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-298750bf-885c-4f8f-9a25-9a969cbcc58c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:11.085536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:11.086529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:11.087558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 160/304] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:11.092903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298750bf-885c-4f8f-9a25-9a969cbcc58c None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:11.126091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:11.180056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f83d50-dc65-4af4-8970-f9f588ae1e12 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:11.188527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:11.216036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298750bf-885c-4f8f-9a25-9a969cbcc58c None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:11.225618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35167cb-2000-4b46-b45a-401a85db82e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:11.226372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35167cb-2000-4b46-b45a-401a85db82e1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.227196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35167cb-2000-4b46-b45a-401a85db82e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.227550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35167cb-2000-4b46-b45a-401a85db82e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.228380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35167cb-2000-4b46-b45a-401a85db82e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:11.229231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35167cb-2000-4b46-b45a-401a85db82e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:11.232710 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35167cb-2000-4b46-b45a-401a85db82e1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:11.245506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 145/305] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:11.251028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35167cb-2000-4b46-b45a-401a85db82e1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:11.256244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac5e94-db9b-41f3-917c-39221eb9e944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:11.257136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac5e94-db9b-41f3-917c-39221eb9e944 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.257545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac5e94-db9b-41f3-917c-39221eb9e944 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.257924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac5e94-db9b-41f3-917c-39221eb9e944 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.258792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac5e94-db9b-41f3-917c-39221eb9e944 None admin] RBAC: Adding query filter params (name=member) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:11.259200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac5e94-db9b-41f3-917c-39221eb9e944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:11.260700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac5e94-db9b-41f3-917c-39221eb9e944 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:11.275825 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 161/306] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:11.276968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac5e94-db9b-41f3-917c-39221eb9e944 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:11.378540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:11.393198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35167cb-2000-4b46-b45a-401a85db82e1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:11.397635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac5e94-db9b-41f3-917c-39221eb9e944 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:11.432311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e522d5-5dc3-48bb-9c0a-c74347112968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:11.433680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e522d5-5dc3-48bb-9c0a-c74347112968 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.433680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e522d5-5dc3-48bb-9c0a-c74347112968 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.433878 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e522d5-5dc3-48bb-9c0a-c74347112968 None admin] PATH_INFO: `/v3/users/6ce9882d360f4476a1b5852ab873e6cd` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.434703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a936bb-32ca-4498-aad4-3e5365551756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:11.435364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a936bb-32ca-4498-aad4-3e5365551756 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.435661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a936bb-32ca-4498-aad4-3e5365551756 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.435944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a936bb-32ca-4498-aad4-3e5365551756 None admin] PATH_INFO: `/v3/users/f3afc34a0afc4194980849cf16767354` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.437521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a936bb-32ca-4498-aad4-3e5365551756 None admin] RBAC: Authorizing `identity:get_user(user_id=f3afc34a0afc4194980849cf16767354)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:11.438684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a936bb-32ca-4498-aad4-3e5365551756 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:11.440332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 162/307] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:11 2022] GET /identity/v3/users/f3afc34a0afc4194980849cf16767354 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:11.446226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b98316-8693-4700-b66c-f8e7015d4f53 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.446226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b98316-8693-4700-b66c-f8e7015d4f53 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.446226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b98316-8693-4700-b66c-f8e7015d4f53 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.446226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 163/308] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:11.447108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b98316-8693-4700-b66c-f8e7015d4f53 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:11.456846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e522d5-5dc3-48bb-9c0a-c74347112968 None admin] RBAC: Authorizing `identity:get_user(user_id=6ce9882d360f4476a1b5852ab873e6cd)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:11.459876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e522d5-5dc3-48bb-9c0a-c74347112968 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:11.462819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 146/309] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:11 2022] GET /identity/v3/users/6ce9882d360f4476a1b5852ab873e6cd => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:11.464726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21388dd5-2dc3-4641-bd62-41b8969d87ff None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.465466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21388dd5-2dc3-4641-bd62-41b8969d87ff None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.465907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21388dd5-2dc3-4641-bd62-41b8969d87ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.490015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-21388dd5-2dc3-4641-bd62-41b8969d87ff None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:11.491870 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-21388dd5-2dc3-4641-bd62-41b8969d87ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:11.518069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:11.559024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b98316-8693-4700-b66c-f8e7015d4f53 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:11.572489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 147/310] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:11.572489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21388dd5-2dc3-4641-bd62-41b8969d87ff None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:11.610829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6db55-0b0a-4b33-a85d-790d1b6a7a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:11.611493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6db55-0b0a-4b33-a85d-790d1b6a7a7d None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.611771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6db55-0b0a-4b33-a85d-790d1b6a7a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.612046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6db55-0b0a-4b33-a85d-790d1b6a7a7d None admin] PATH_INFO: `/v3/system/users/f3afc34a0afc4194980849cf16767354/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.613654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6db55-0b0a-4b33-a85d-790d1b6a7a7d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f3afc34a0afc4194980849cf16767354, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:11.615157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6db55-0b0a-4b33-a85d-790d1b6a7a7d None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:11.678282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21388dd5-2dc3-4641-bd62-41b8969d87ff None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:11.713478 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd85710-a3f5-4037-92b1-51ecd0ece96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:11.714105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd85710-a3f5-4037-92b1-51ecd0ece96d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.714394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd85710-a3f5-4037-92b1-51ecd0ece96d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.714672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd85710-a3f5-4037-92b1-51ecd0ece96d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.715477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd85710-a3f5-4037-92b1-51ecd0ece96d None admin] RBAC: Adding query filter params (role.id=8e94dcc6eec7482fab6c8aba670a4b6c, user.id=6ce9882d360f4476a1b5852ab873e6cd, scope.system=all) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:11.715794 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd85710-a3f5-4037-92b1-51ecd0ece96d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:11.717251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd85710-a3f5-4037-92b1-51ecd0ece96d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:11.717857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 164/311] 10.222.0.69 () {58 vars in 1334 bytes} [Thu Jul 28 06:09:11 2022] PUT /identity/v3/system/users/f3afc34a0afc4194980849cf16767354/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:11.731360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 148/312] 10.222.0.69 () {56 vars in 1308 bytes} [Thu Jul 28 06:09:11 2022] GET /identity/v3/role_assignments?user.id=6ce9882d360f4476a1b5852ab873e6cd&scope.system=all&role.id=8e94dcc6eec7482fab6c8aba670a4b6c => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:11.731587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6db55-0b0a-4b33-a85d-790d1b6a7a7d None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:11.763360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:11.866543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6db55-0b0a-4b33-a85d-790d1b6a7a7d None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:11.914020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87cc27-e60a-4957-8322-ae5f54f6b5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:11.914694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87cc27-e60a-4957-8322-ae5f54f6b5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:11.914983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87cc27-e60a-4957-8322-ae5f54f6b5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:11.915268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87cc27-e60a-4957-8322-ae5f54f6b5c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:11.926760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87cc27-e60a-4957-8322-ae5f54f6b5c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-3f87cc27-e60a-4957-8322-ae5f54f6b5c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:11.932172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:11.933016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:11.933831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 165/313] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:11.939515 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd85710-a3f5-4037-92b1-51ecd0ece96d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:11.963926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:12.051532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd85710-a3f5-4037-92b1-51ecd0ece96d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:12.093463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42590b3a-85b6-43e6-b9a6-1a995d510d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:12.094114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42590b3a-85b6-43e6-b9a6-1a995d510d8f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:12.094399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42590b3a-85b6-43e6-b9a6-1a995d510d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:12.094679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42590b3a-85b6-43e6-b9a6-1a995d510d8f None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:12.095436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42590b3a-85b6-43e6-b9a6-1a995d510d8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:12.095748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42590b3a-85b6-43e6-b9a6-1a995d510d8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:12.097217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42590b3a-85b6-43e6-b9a6-1a995d510d8f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:12.107499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 149/314] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:12.113142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87cc27-e60a-4957-8322-ae5f54f6b5c1 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:12.204082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:12.227528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87cc27-e60a-4957-8322-ae5f54f6b5c1 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:12.265213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692ebe5-c124-458e-9a9c-7333948fe745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:12.266255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692ebe5-c124-458e-9a9c-7333948fe745 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:12.266737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692ebe5-c124-458e-9a9c-7333948fe745 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:12.267210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692ebe5-c124-458e-9a9c-7333948fe745 None admin] PATH_INFO: `/v3/users/261d0a584b394fd8add4e99cab7c07a3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:12.268586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692ebe5-c124-458e-9a9c-7333948fe745 None admin] RBAC: Authorizing `identity:get_user(user_id=261d0a584b394fd8add4e99cab7c07a3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:12.270441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692ebe5-c124-458e-9a9c-7333948fe745 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:12.272885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 166/315] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:12 2022] GET /identity/v3/users/261d0a584b394fd8add4e99cab7c07a3 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:12.278241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42590b3a-85b6-43e6-b9a6-1a995d510d8f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:12.367288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:12.385988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42590b3a-85b6-43e6-b9a6-1a995d510d8f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:12.423176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2975db-b4de-4dd5-ba00-fe83205b304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:12.423803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2975db-b4de-4dd5-ba00-fe83205b304f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:12.424085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2975db-b4de-4dd5-ba00-fe83205b304f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:12.424362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2975db-b4de-4dd5-ba00-fe83205b304f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:12.497460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2975db-b4de-4dd5-ba00-fe83205b304f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:12.498938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2975db-b4de-4dd5-ba00-fe83205b304f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:12.587662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:12.607603 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 150/316] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:12.613174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692ebe5-c124-458e-9a9c-7333948fe745 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:12.718805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692ebe5-c124-458e-9a9c-7333948fe745 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:12.755536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5063543-c2be-4325-93c1-7491111f3fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:12.756526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5063543-c2be-4325-93c1-7491111f3fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:12.757035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5063543-c2be-4325-93c1-7491111f3fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:12.757552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5063543-c2be-4325-93c1-7491111f3fa7 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:12.758928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5063543-c2be-4325-93c1-7491111f3fa7 None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:12.760683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5063543-c2be-4325-93c1-7491111f3fa7 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:12.762908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 167/317] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:12 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:12.769222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2975db-b4de-4dd5-ba00-fe83205b304f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:12.873950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2975db-b4de-4dd5-ba00-fe83205b304f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:12.913185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac535fe-dacf-42a7-bafa-37e0253d6b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:12.913838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac535fe-dacf-42a7-bafa-37e0253d6b32 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:12.914120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac535fe-dacf-42a7-bafa-37e0253d6b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:12.914394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac535fe-dacf-42a7-bafa-37e0253d6b32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:12.915198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac535fe-dacf-42a7-bafa-37e0253d6b32 None admin] RBAC: Adding query filter params (role.id=8e94dcc6eec7482fab6c8aba670a4b6c, scope.project.id=0660a31d921d44188bb944fcdbb9b027, user.id=261d0a584b394fd8add4e99cab7c07a3) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:12.915506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac535fe-dacf-42a7-bafa-37e0253d6b32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:12.916946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac535fe-dacf-42a7-bafa-37e0253d6b32 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:12.923531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 151/318] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:12 2022] GET /identity/v3/role_assignments?user.id=261d0a584b394fd8add4e99cab7c07a3&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=8e94dcc6eec7482fab6c8aba670a4b6c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:13.075496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923658e-c737-4805-b6bd-06fbc11ab043 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:13.076117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923658e-c737-4805-b6bd-06fbc11ab043 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:13.076623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923658e-c737-4805-b6bd-06fbc11ab043 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:13.078628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 168/319] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:13.085618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbca0c-6aad-400c-a39d-234ce56c22a6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:13.085618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbca0c-6aad-400c-a39d-234ce56c22a6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:13.085618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbca0c-6aad-400c-a39d-234ce56c22a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:13.103916 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bbbbca0c-6aad-400c-a39d-234ce56c22a6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:13.105123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bbbbca0c-6aad-400c-a39d-234ce56c22a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:13.154436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:13.182399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 152/320] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:13.205918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923658e-c737-4805-b6bd-06fbc11ab043 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:13.222335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:13.314271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923658e-c737-4805-b6bd-06fbc11ab043 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:13.355009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da346ba-d4ec-4237-92ac-de83d1bcc598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:13.355997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da346ba-d4ec-4237-92ac-de83d1bcc598 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:13.356496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da346ba-d4ec-4237-92ac-de83d1bcc598 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:13.356977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da346ba-d4ec-4237-92ac-de83d1bcc598 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:13.358388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da346ba-d4ec-4237-92ac-de83d1bcc598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:13.360030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da346ba-d4ec-4237-92ac-de83d1bcc598 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:13.362760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 169/321] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:13.368728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbca0c-6aad-400c-a39d-234ce56c22a6 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:13.483169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbca0c-6aad-400c-a39d-234ce56c22a6 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:13.561156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6c480-c27a-48e5-8aa0-e7c7d8622d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:13.561827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6c480-c27a-48e5-8aa0-e7c7d8622d7a None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:13.562111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6c480-c27a-48e5-8aa0-e7c7d8622d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:13.562384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6c480-c27a-48e5-8aa0-e7c7d8622d7a None admin] PATH_INFO: `/v3/groups` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:13.563054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6c480-c27a-48e5-8aa0-e7c7d8622d7a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:13.564556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6c480-c27a-48e5-8aa0-e7c7d8622d7a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:13.632039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 153/322] 10.222.0.69 () {60 vars in 1134 bytes} [Thu Jul 28 06:09:13 2022] POST /identity/v3/groups => generated 226 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:09:13.849280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba603ab-212b-472e-94c4-50961ae8a58b None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:13.849887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba603ab-212b-472e-94c4-50961ae8a58b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:13.850177 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba603ab-212b-472e-94c4-50961ae8a58b None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:13.852173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 170/323] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:13.861350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206eba1-0009-45af-80b7-f4412723f27b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:13.861765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206eba1-0009-45af-80b7-f4412723f27b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:13.861922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206eba1-0009-45af-80b7-f4412723f27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:13.885364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c206eba1-0009-45af-80b7-f4412723f27b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:13.886549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c206eba1-0009-45af-80b7-f4412723f27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:13.918896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:13.935724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eaf957-b739-47e1-97c0-21199c2c7607 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:13.936064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eaf957-b739-47e1-97c0-21199c2c7607 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:13.936283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eaf957-b739-47e1-97c0-21199c2c7607 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:13.940296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 171/324] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:09:13.944337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98219-94db-4f28-ac82-3102a38788d0 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:13.944337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98219-94db-4f28-ac82-3102a38788d0 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:13.944831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98219-94db-4f28-ac82-3102a38788d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:13.968696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-7bb98219-94db-4f28-ac82-3102a38788d0 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:13.970037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7bb98219-94db-4f28-ac82-3102a38788d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:13.971938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 154/325] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:13.997280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206eba1-0009-45af-80b7-f4412723f27b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:14.071384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 172/326] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:14.092884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98219-94db-4f28-ac82-3102a38788d0 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:14.113700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:14.126739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206eba1-0009-45af-80b7-f4412723f27b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:14.184225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ac653-b516-4880-814d-df06404b4f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:14.184808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ac653-b516-4880-814d-df06404b4f4c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:14.185094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ac653-b516-4880-814d-df06404b4f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:14.185401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ac653-b516-4880-814d-df06404b4f4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:14.198470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ac653-b516-4880-814d-df06404b4f4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a17ac653-b516-4880-814d-df06404b4f4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:14.204703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:14.205610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:14.206395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 155/327] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:14.214455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ac653-b516-4880-814d-df06404b4f4c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:14.218817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98219-94db-4f28-ac82-3102a38788d0 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:14.253683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:14.262371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fec5f-4321-40a1-9798-5bc79352b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:14.263033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fec5f-4321-40a1-9798-5bc79352b4ab None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:14.263916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fec5f-4321-40a1-9798-5bc79352b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:14.264497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fec5f-4321-40a1-9798-5bc79352b4ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:14.278922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fec5f-4321-40a1-9798-5bc79352b4ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-984fec5f-4321-40a1-9798-5bc79352b4ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:14.286466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:14.287503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:14.288287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:14.289126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 173/328] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:14.294583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fec5f-4321-40a1-9798-5bc79352b4ab None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:14.347335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ac653-b516-4880-814d-df06404b4f4c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:14.394902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:14.396025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c143fef-cddb-445e-b870-ea78567ffc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:14.396679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c143fef-cddb-445e-b870-ea78567ffc7c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:14.396969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c143fef-cddb-445e-b870-ea78567ffc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:14.397250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c143fef-cddb-445e-b870-ea78567ffc7c None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:14.398038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c143fef-cddb-445e-b870-ea78567ffc7c None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:14.398353 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c143fef-cddb-445e-b870-ea78567ffc7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:14.399768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c143fef-cddb-445e-b870-ea78567ffc7c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:14.412081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 156/329] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:14.418347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c143fef-cddb-445e-b870-ea78567ffc7c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:14.437088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fec5f-4321-40a1-9798-5bc79352b4ab None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:14.486140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93271405-6974-41ef-8b16-66e9ee238f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:14.487717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93271405-6974-41ef-8b16-66e9ee238f3b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:14.488190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93271405-6974-41ef-8b16-66e9ee238f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:14.488610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93271405-6974-41ef-8b16-66e9ee238f3b None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:14.489996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93271405-6974-41ef-8b16-66e9ee238f3b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:14.490446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93271405-6974-41ef-8b16-66e9ee238f3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:14.492125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93271405-6974-41ef-8b16-66e9ee238f3b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:14.505698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 174/330] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:14.511729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93271405-6974-41ef-8b16-66e9ee238f3b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:14.535315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:14.549115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c143fef-cddb-445e-b870-ea78567ffc7c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:14.594096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb328567-3ff4-43d9-a547-19cc1bf77ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:14.594956 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb328567-3ff4-43d9-a547-19cc1bf77ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:14.595311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb328567-3ff4-43d9-a547-19cc1bf77ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:14.595671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb328567-3ff4-43d9-a547-19cc1bf77ac7 None admin] PATH_INFO: `/v3/users/f3afc34a0afc4194980849cf16767354` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:14.596951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb328567-3ff4-43d9-a547-19cc1bf77ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=f3afc34a0afc4194980849cf16767354)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:14.599113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb328567-3ff4-43d9-a547-19cc1bf77ac7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:14.601344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 157/331] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:14 2022] GET /identity/v3/users/f3afc34a0afc4194980849cf16767354 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:14.607088 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb328567-3ff4-43d9-a547-19cc1bf77ac7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:14.648011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93271405-6974-41ef-8b16-66e9ee238f3b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:14.692385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fe058-64cc-41d7-84ca-5f94b3be3288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:14.694433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fe058-64cc-41d7-84ca-5f94b3be3288 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:14.694993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fe058-64cc-41d7-84ca-5f94b3be3288 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:14.695444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fe058-64cc-41d7-84ca-5f94b3be3288 None admin] PATH_INFO: `/v3/users/6ce9882d360f4476a1b5852ab873e6cd` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:14.697113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fe058-64cc-41d7-84ca-5f94b3be3288 None admin] RBAC: Authorizing `identity:get_user(user_id=6ce9882d360f4476a1b5852ab873e6cd)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:14.698809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fe058-64cc-41d7-84ca-5f94b3be3288 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:14.702513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 175/332] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:14 2022] GET /identity/v3/users/6ce9882d360f4476a1b5852ab873e6cd => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:14.709929 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fe058-64cc-41d7-84ca-5f94b3be3288 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:14.733682 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:14.751140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb328567-3ff4-43d9-a547-19cc1bf77ac7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:14.798174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc7883-6a11-421d-af35-41d7ea14df74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:14.798849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc7883-6a11-421d-af35-41d7ea14df74 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:14.799143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc7883-6a11-421d-af35-41d7ea14df74 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:14.799428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc7883-6a11-421d-af35-41d7ea14df74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:14.806324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc7883-6a11-421d-af35-41d7ea14df74 None admin] RBAC: Adding query filter params (role.id=c4fc73849d664b2588bd2de7372aa839, user.id=f3afc34a0afc4194980849cf16767354, scope.system=all) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:14.806324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc7883-6a11-421d-af35-41d7ea14df74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:14.806324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc7883-6a11-421d-af35-41d7ea14df74 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:14.818295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 158/333] 10.222.0.69 () {56 vars in 1308 bytes} [Thu Jul 28 06:09:14 2022] GET /identity/v3/role_assignments?user.id=f3afc34a0afc4194980849cf16767354&scope.system=all&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:14.898275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fe058-64cc-41d7-84ca-5f94b3be3288 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:14.945523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba7360-d40a-4ba8-ae1a-27c057f5cb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:14.946137 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba7360-d40a-4ba8-ae1a-27c057f5cb66 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:14.946430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba7360-d40a-4ba8-ae1a-27c057f5cb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:14.946714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba7360-d40a-4ba8-ae1a-27c057f5cb66 None admin] PATH_INFO: `/v3/system/users/6ce9882d360f4476a1b5852ab873e6cd/roles/8e94dcc6eec7482fab6c8aba670a4b6c` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:14.954286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba7360-d40a-4ba8-ae1a-27c057f5cb66 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6ce9882d360f4476a1b5852ab873e6cd, role_id=8e94dcc6eec7482fab6c8aba670a4b6c)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:14.955727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba7360-d40a-4ba8-ae1a-27c057f5cb66 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:15.014524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 176/334] 10.222.0.69 () {58 vars in 1334 bytes} [Thu Jul 28 06:09:14 2022] PUT /identity/v3/system/users/6ce9882d360f4476a1b5852ab873e6cd/roles/8e94dcc6eec7482fab6c8aba670a4b6c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:16.696306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95ec2e-09c0-4943-9ac0-3edcace47694 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:16.696306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95ec2e-09c0-4943-9ac0-3edcace47694 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:16.696306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95ec2e-09c0-4943-9ac0-3edcace47694 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:16.697495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 159/335] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:16.704010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d2634-ff69-4da4-a2dc-fcd488db4b56 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:16.704866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d2634-ff69-4da4-a2dc-fcd488db4b56 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:16.705380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d2634-ff69-4da4-a2dc-fcd488db4b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:16.725210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-153d2634-ff69-4da4-a2dc-fcd488db4b56 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:16.726772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-153d2634-ff69-4da4-a2dc-fcd488db4b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:16.739368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bd34c-2856-4fdf-904c-bbc2c69ebd85 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:16.740352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bd34c-2856-4fdf-904c-bbc2c69ebd85 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:16.740866 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bd34c-2856-4fdf-904c-bbc2c69ebd85 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:16.742771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 160/336] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:16.756080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7849c6-4bcb-42c9-a0a0-5340d251cb6e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:16.756713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7849c6-4bcb-42c9-a0a0-5340d251cb6e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:16.757216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7849c6-4bcb-42c9-a0a0-5340d251cb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:16.799583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ed7849c6-4bcb-42c9-a0a0-5340d251cb6e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:16.801732 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ed7849c6-4bcb-42c9-a0a0-5340d251cb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:16.824074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:16.849695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:16.867533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 177/337] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:16.898583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d2634-ff69-4da4-a2dc-fcd488db4b56 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:16.933447 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 161/338] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:16.938155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da871aa-a120-4973-b267-595e01a7d504 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:16.938455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da871aa-a120-4973-b267-595e01a7d504 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:16.938726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da871aa-a120-4973-b267-595e01a7d504 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:16.940537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 162/339] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:16.946468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ee272-0820-4a4e-b691-629786e0e338 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:16.947032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ee272-0820-4a4e-b691-629786e0e338 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:16.947463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ee272-0820-4a4e-b691-629786e0e338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:16.980575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-812ee272-0820-4a4e-b691-629786e0e338 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:16.982026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-812ee272-0820-4a4e-b691-629786e0e338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:17.018894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d2634-ff69-4da4-a2dc-fcd488db4b56 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:17.065615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c9600-0418-4307-9041-956b2b72726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:17.066280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c9600-0418-4307-9041-956b2b72726e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:17.066562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c9600-0418-4307-9041-956b2b72726e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:17.067451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c9600-0418-4307-9041-956b2b72726e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:17.087370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c9600-0418-4307-9041-956b2b72726e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-a68c9600-0418-4307-9041-956b2b72726e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:17.097774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:17.099495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:17.100493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:17.105890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 178/340] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:09:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:17.107134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c9600-0418-4307-9041-956b2b72726e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:17.125903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 163/341] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:17.127177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ee272-0820-4a4e-b691-629786e0e338 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:17.230458 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:17.252738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:17.295803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c9600-0418-4307-9041-956b2b72726e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:17.305180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ee272-0820-4a4e-b691-629786e0e338 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:17.336241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090632-e463-43c0-bb19-785cb243a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:17.337206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090632-e463-43c0-bb19-785cb243a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:17.337710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090632-e463-43c0-bb19-785cb243a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:17.338120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090632-e463-43c0-bb19-785cb243a7c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:17.349670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c090632-e463-43c0-bb19-785cb243a7c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-6c090632-e463-43c0-bb19-785cb243a7c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:17.357959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:17.359112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:17.360094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:17.362198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 179/342] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:17.363365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090632-e463-43c0-bb19-785cb243a7c3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:17.368015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc00ad8-df68-4e7f-ade1-20469e73e614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:17.368889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc00ad8-df68-4e7f-ade1-20469e73e614 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:17.369462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc00ad8-df68-4e7f-ade1-20469e73e614 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:17.369962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc00ad8-df68-4e7f-ade1-20469e73e614 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:17.370911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc00ad8-df68-4e7f-ade1-20469e73e614 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:17.371442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc00ad8-df68-4e7f-ade1-20469e73e614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:17.373081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc00ad8-df68-4e7f-ade1-20469e73e614 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:17.387329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 164/343] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:17.388537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc00ad8-df68-4e7f-ade1-20469e73e614 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:17.481179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090632-e463-43c0-bb19-785cb243a7c3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:17.498938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:17.517990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79eaf4-a7dd-4f52-bc88-edd29f1a2abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:17.518920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79eaf4-a7dd-4f52-bc88-edd29f1a2abf None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:17.519368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79eaf4-a7dd-4f52-bc88-edd29f1a2abf None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:17.519821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79eaf4-a7dd-4f52-bc88-edd29f1a2abf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:17.531039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79eaf4-a7dd-4f52-bc88-edd29f1a2abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-eb79eaf4-a7dd-4f52-bc88-edd29f1a2abf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:17.537856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:17.538981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:17.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:17.542361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 180/344] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:17.543361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79eaf4-a7dd-4f52-bc88-edd29f1a2abf None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:17.592340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc00ad8-df68-4e7f-ade1-20469e73e614 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:17.649104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79eaf4-a7dd-4f52-bc88-edd29f1a2abf None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:17.667042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96058aa-5fc8-4d81-8dbf-d9680cd62f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:17.667796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96058aa-5fc8-4d81-8dbf-d9680cd62f55 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:17.668267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96058aa-5fc8-4d81-8dbf-d9680cd62f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:17.668595 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96058aa-5fc8-4d81-8dbf-d9680cd62f55 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:17.669667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96058aa-5fc8-4d81-8dbf-d9680cd62f55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:17.670048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96058aa-5fc8-4d81-8dbf-d9680cd62f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:17.671546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96058aa-5fc8-4d81-8dbf-d9680cd62f55 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:17.678675 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:17.688508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd13d6f-4223-40fd-a99b-34fd1febe820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:17.691758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd13d6f-4223-40fd-a99b-34fd1febe820 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:17.691758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd13d6f-4223-40fd-a99b-34fd1febe820 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:17.691758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd13d6f-4223-40fd-a99b-34fd1febe820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:17.692010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 165/345] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:17.692010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96058aa-5fc8-4d81-8dbf-d9680cd62f55 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:17.739551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:17.763402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd13d6f-4223-40fd-a99b-34fd1febe820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:17.765476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd13d6f-4223-40fd-a99b-34fd1febe820 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:17.881416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 181/346] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:17.882651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd13d6f-4223-40fd-a99b-34fd1febe820 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:17.902323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96058aa-5fc8-4d81-8dbf-d9680cd62f55 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:17.967322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:17.968587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb85d03-6274-4603-9d30-294c4a83e5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:17.969538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb85d03-6274-4603-9d30-294c4a83e5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:17.969968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb85d03-6274-4603-9d30-294c4a83e5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:17.970417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb85d03-6274-4603-9d30-294c4a83e5d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:17.971338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb85d03-6274-4603-9d30-294c4a83e5d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:17.971791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb85d03-6274-4603-9d30-294c4a83e5d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:17.987784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb85d03-6274-4603-9d30-294c4a83e5d1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:17.994887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd13d6f-4223-40fd-a99b-34fd1febe820 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:17.996263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 166/347] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:17.997348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb85d03-6274-4603-9d30-294c4a83e5d1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:18.032919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e66593-b48d-45c5-85be-9a039620ec0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:18.033630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66593-b48d-45c5-85be-9a039620ec0a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.033915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66593-b48d-45c5-85be-9a039620ec0a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.034190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66593-b48d-45c5-85be-9a039620ec0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.112143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e66593-b48d-45c5-85be-9a039620ec0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:18.114225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e66593-b48d-45c5-85be-9a039620ec0a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:18.130140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:18.218413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:18.223258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 182/348] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:18.224653 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e66593-b48d-45c5-85be-9a039620ec0a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:18.247506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb85d03-6274-4603-9d30-294c4a83e5d1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:18.322421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae0074-194c-4948-95b9-49eb764c1be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:18.323147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae0074-194c-4948-95b9-49eb764c1be0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.323437 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae0074-194c-4948-95b9-49eb764c1be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.323716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae0074-194c-4948-95b9-49eb764c1be0 None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.325818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae0074-194c-4948-95b9-49eb764c1be0 None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:18.327315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae0074-194c-4948-95b9-49eb764c1be0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:18.329227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 167/349] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:17 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:18.329925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae0074-194c-4948-95b9-49eb764c1be0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:18.444293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e66593-b48d-45c5-85be-9a039620ec0a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:18.477104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae0074-194c-4948-95b9-49eb764c1be0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:18.490399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b66ff-98a6-4034-98eb-2b7f138e1c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:18.491033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b66ff-98a6-4034-98eb-2b7f138e1c6a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.491317 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b66ff-98a6-4034-98eb-2b7f138e1c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.491596 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b66ff-98a6-4034-98eb-2b7f138e1c6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.508557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:18.524522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedda74b-3611-4204-a0e3-2fc9c2203a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:18.525281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedda74b-3611-4204-a0e3-2fc9c2203a3a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.525648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedda74b-3611-4204-a0e3-2fc9c2203a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.526046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedda74b-3611-4204-a0e3-2fc9c2203a3a None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.527212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedda74b-3611-4204-a0e3-2fc9c2203a3a None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:18.528685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedda74b-3611-4204-a0e3-2fc9c2203a3a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:18.530403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 168/350] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:18 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:18.531281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedda74b-3611-4204-a0e3-2fc9c2203a3a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:18.551872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:18.583482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b66ff-98a6-4034-98eb-2b7f138e1c6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:18.584986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b66ff-98a6-4034-98eb-2b7f138e1c6a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:18.657915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedda74b-3611-4204-a0e3-2fc9c2203a3a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:18.701516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0e922-666b-439b-99f2-eb83aa4ca0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:18.702189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e922-666b-439b-99f2-eb83aa4ca0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.702480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e922-666b-439b-99f2-eb83aa4ca0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.702769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e922-666b-439b-99f2-eb83aa4ca0a4 None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.710174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0e922-666b-439b-99f2-eb83aa4ca0a4 None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:18.711947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0e922-666b-439b-99f2-eb83aa4ca0a4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:18.713564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 169/351] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:18 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:18.714748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd7e20-ba4a-4c0f-b716-2b89e9390d8f None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.715041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd7e20-ba4a-4c0f-b716-2b89e9390d8f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.715324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd7e20-ba4a-4c0f-b716-2b89e9390d8f None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.716548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 170/352] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:18.718785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1ff2-d8b7-4ffd-afe1-d35033ad356e None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.719346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1ff2-d8b7-4ffd-afe1-d35033ad356e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.719634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1ff2-d8b7-4ffd-afe1-d35033ad356e None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.719748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 183/353] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:18.721049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf219f-5900-49c9-9d06-74d79142cad4 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.721495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf219f-5900-49c9-9d06-74d79142cad4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.721666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf219f-5900-49c9-9d06-74d79142cad4 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.723546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 184/355] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:18.725455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 171/355] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:18.726409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1ff2-d8b7-4ffd-afe1-d35033ad356e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:18.728505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf219f-5900-49c9-9d06-74d79142cad4 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:18.800657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:18.828731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:18.851641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1ff2-d8b7-4ffd-afe1-d35033ad356e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:18.857598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf219f-5900-49c9-9d06-74d79142cad4 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:18.898071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356f703-cb85-4b89-b9cf-d5bf6b5602d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:18.898713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356f703-cb85-4b89-b9cf-d5bf6b5602d3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.899000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356f703-cb85-4b89-b9cf-d5bf6b5602d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.899282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356f703-cb85-4b89-b9cf-d5bf6b5602d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.900081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356f703-cb85-4b89-b9cf-d5bf6b5602d3 None admin] RBAC: Adding query filter params (group.id=72ec25c95e614ac8b7558060c5e30674, role.id=c7b4bfc82f3b46cfaa10773dad7cd03a, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:18.900397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356f703-cb85-4b89-b9cf-d5bf6b5602d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:18.902065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26beb0-0ec1-4192-a8cf-ccbf4a2967aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:18.902723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26beb0-0ec1-4192-a8cf-ccbf4a2967aa None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.902821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356f703-cb85-4b89-b9cf-d5bf6b5602d3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:18.903253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26beb0-0ec1-4192-a8cf-ccbf4a2967aa None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.903253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26beb0-0ec1-4192-a8cf-ccbf4a2967aa None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.904915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26beb0-0ec1-4192-a8cf-ccbf4a2967aa None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:18.906445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26beb0-0ec1-4192-a8cf-ccbf4a2967aa None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:18.908128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 185/356] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:18 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:18.909447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e755a-3f6f-4cff-9ffb-12af0741a138 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:18.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e755a-3f6f-4cff-9ffb-12af0741a138 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:18.910021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e755a-3f6f-4cff-9ffb-12af0741a138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:18.912527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 172/357] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 28 06:09:18 2022] GET /identity/v3/role_assignments?group.id=72ec25c95e614ac8b7558060c5e30674&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=c7b4bfc82f3b46cfaa10773dad7cd03a => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:18.914653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356f703-cb85-4b89-b9cf-d5bf6b5602d3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:18.934329 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-731e755a-3f6f-4cff-9ffb-12af0741a138 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:18.935569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-731e755a-3f6f-4cff-9ffb-12af0741a138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:19.039951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 186/358] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:19.042258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e43472-693a-4473-b509-12139c843a48 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:19.042778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e43472-693a-4473-b509-12139c843a48 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:19.043286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e43472-693a-4473-b509-12139c843a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:19.064296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356f703-cb85-4b89-b9cf-d5bf6b5602d3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:19.064979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-96e43472-693a-4473-b509-12139c843a48 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:19.066476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-96e43472-693a-4473-b509-12139c843a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:19.096722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:19.106347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d848b5-3a36-45fd-b401-2a0f066a8f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:19.106970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d848b5-3a36-45fd-b401-2a0f066a8f4e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:19.113251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d848b5-3a36-45fd-b401-2a0f066a8f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:19.113251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d848b5-3a36-45fd-b401-2a0f066a8f4e None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:19.113251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d848b5-3a36-45fd-b401-2a0f066a8f4e None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:19.113251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d848b5-3a36-45fd-b401-2a0f066a8f4e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:19.113251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 173/359] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:18 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:19.113251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adcffa-b3e5-40d1-8a8b-9ba781b17f63 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:19.114297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adcffa-b3e5-40d1-8a8b-9ba781b17f63 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:19.114297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adcffa-b3e5-40d1-8a8b-9ba781b17f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:19.135517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-48adcffa-b3e5-40d1-8a8b-9ba781b17f63 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:19.136703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-48adcffa-b3e5-40d1-8a8b-9ba781b17f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:19.151576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 187/360] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:19.152436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e43472-693a-4473-b509-12139c843a48 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:19.157147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:19.238185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 174/361] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:19.239448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48adcffa-b3e5-40d1-8a8b-9ba781b17f63 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:19.298499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e43472-693a-4473-b509-12139c843a48 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:19.344110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab644f08-c63e-4f15-823b-4013d736c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:19.344679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab644f08-c63e-4f15-823b-4013d736c948 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:19.345489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab644f08-c63e-4f15-823b-4013d736c948 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:19.345784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab644f08-c63e-4f15-823b-4013d736c948 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:19.348977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab644f08-c63e-4f15-823b-4013d736c948 None admin] RBAC: Adding query filter params (group.id=72ec25c95e614ac8b7558060c5e30674, role.id=c4fc73849d664b2588bd2de7372aa839, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:19.348977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab644f08-c63e-4f15-823b-4013d736c948 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:19.349886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab644f08-c63e-4f15-823b-4013d736c948 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:19.358203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 188/362] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 28 06:09:19 2022] GET /identity/v3/role_assignments?group.id=72ec25c95e614ac8b7558060c5e30674&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:19.358680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab644f08-c63e-4f15-823b-4013d736c948 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:19.388118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48adcffa-b3e5-40d1-8a8b-9ba781b17f63 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:19.433267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:19.455370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f24f8-caa6-4880-9439-8fa67b4d9c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:19.461276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f24f8-caa6-4880-9439-8fa67b4d9c33 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:19.461276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f24f8-caa6-4880-9439-8fa67b4d9c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:19.461276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f24f8-caa6-4880-9439-8fa67b4d9c33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:19.475179 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f24f8-caa6-4880-9439-8fa67b4d9c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:19.475821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8a6f24f8-caa6-4880-9439-8fa67b4d9c33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:19.481146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:19.482158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 175/363] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:09:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:19.482924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f24f8-caa6-4880-9439-8fa67b4d9c33 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:19.527629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab644f08-c63e-4f15-823b-4013d736c948 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:19.573008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd0dde-b2f8-41b7-bb15-3f975b265469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:19.573697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd0dde-b2f8-41b7-bb15-3f975b265469 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:19.573980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd0dde-b2f8-41b7-bb15-3f975b265469 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:19.574299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd0dde-b2f8-41b7-bb15-3f975b265469 None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:19.575488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd0dde-b2f8-41b7-bb15-3f975b265469 None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:19.576923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd0dde-b2f8-41b7-bb15-3f975b265469 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:19.579061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 189/364] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:19 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:19.579640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd0dde-b2f8-41b7-bb15-3f975b265469 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:19.616756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f24f8-caa6-4880-9439-8fa67b4d9c33 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:19.655453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456d65f-bcd9-4c94-9232-f8faf5ec061d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:19.656492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d65f-bcd9-4c94-9232-f8faf5ec061d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:19.656985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d65f-bcd9-4c94-9232-f8faf5ec061d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:19.658069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d65f-bcd9-4c94-9232-f8faf5ec061d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:19.669836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456d65f-bcd9-4c94-9232-f8faf5ec061d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-1456d65f-bcd9-4c94-9232-f8faf5ec061d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:19.675142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:19.676107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:19.677099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:19.679834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 176/365] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:09:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:19.680861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456d65f-bcd9-4c94-9232-f8faf5ec061d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:19.697439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd0dde-b2f8-41b7-bb15-3f975b265469 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:19.737693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:19.739578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e449144-e31f-4d3b-9179-48cb72822e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:19.740227 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e449144-e31f-4d3b-9179-48cb72822e33 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:19.740515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e449144-e31f-4d3b-9179-48cb72822e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:19.740795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e449144-e31f-4d3b-9179-48cb72822e33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:19.746774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:19.761035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e449144-e31f-4d3b-9179-48cb72822e33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-7e449144-e31f-4d3b-9179-48cb72822e33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:19.766229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:19.767174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:19.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 190/366] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:19.769131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e449144-e31f-4d3b-9179-48cb72822e33 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:19.814277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456d65f-bcd9-4c94-9232-f8faf5ec061d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:19.869159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166216b-a7b4-4810-803b-02edf9788dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:19.869752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166216b-a7b4-4810-803b-02edf9788dbb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:19.870043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166216b-a7b4-4810-803b-02edf9788dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:19.870325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166216b-a7b4-4810-803b-02edf9788dbb None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:19.871058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166216b-a7b4-4810-803b-02edf9788dbb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:19.871369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166216b-a7b4-4810-803b-02edf9788dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:19.875646 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166216b-a7b4-4810-803b-02edf9788dbb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:19.885734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 177/367] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:19.886382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166216b-a7b4-4810-803b-02edf9788dbb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:19.910002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e449144-e31f-4d3b-9179-48cb72822e33 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:19.954933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30de391-9f6f-4149-b3ff-9094cfd3985f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:19.955816 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30de391-9f6f-4149-b3ff-9094cfd3985f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:19.956102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30de391-9f6f-4149-b3ff-9094cfd3985f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:19.956619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30de391-9f6f-4149-b3ff-9094cfd3985f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:19.958485 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30de391-9f6f-4149-b3ff-9094cfd3985f None admin] RBAC: Adding query filter params (group.id=72ec25c95e614ac8b7558060c5e30674, role.id=c4fc73849d664b2588bd2de7372aa839, scope.project.id=0660a31d921d44188bb944fcdbb9b027) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:19.958817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30de391-9f6f-4149-b3ff-9094cfd3985f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:19.960271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30de391-9f6f-4149-b3ff-9094cfd3985f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:19.968761 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 191/368] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 28 06:09:19 2022] GET /identity/v3/role_assignments?group.id=72ec25c95e614ac8b7558060c5e30674&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:19.973449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30de391-9f6f-4149-b3ff-9094cfd3985f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:20.038625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166216b-a7b4-4810-803b-02edf9788dbb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:20.045268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:20.121649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f6d2b-5ad8-412d-966b-23a3b4ea5415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:20.122225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f6d2b-5ad8-412d-966b-23a3b4ea5415 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:20.122514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f6d2b-5ad8-412d-966b-23a3b4ea5415 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:20.123130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f6d2b-5ad8-412d-966b-23a3b4ea5415 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:20.124369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:20.125149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f6d2b-5ad8-412d-966b-23a3b4ea5415 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:20.125491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f6d2b-5ad8-412d-966b-23a3b4ea5415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:20.127128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f6d2b-5ad8-412d-966b-23a3b4ea5415 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:20.138298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 178/369] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:20.139612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f6d2b-5ad8-412d-966b-23a3b4ea5415 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:20.215580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30de391-9f6f-4149-b3ff-9094cfd3985f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:20.261362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe29eb-7aa4-4887-b558-a9465ee16dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:20.262737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe29eb-7aa4-4887-b558-a9465ee16dfd None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:20.263110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe29eb-7aa4-4887-b558-a9465ee16dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:20.263301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe29eb-7aa4-4887-b558-a9465ee16dfd None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:20.264304 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe29eb-7aa4-4887-b558-a9465ee16dfd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:20.264648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe29eb-7aa4-4887-b558-a9465ee16dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:20.266074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe29eb-7aa4-4887-b558-a9465ee16dfd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:20.275851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 192/370] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:20.276810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe29eb-7aa4-4887-b558-a9465ee16dfd None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:20.288820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f6d2b-5ad8-412d-966b-23a3b4ea5415 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:20.337830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d3f24-31dd-449e-9428-eb41a4200c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:20.338953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3f24-31dd-449e-9428-eb41a4200c82 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:20.339435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3f24-31dd-449e-9428-eb41a4200c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:20.339907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3f24-31dd-449e-9428-eb41a4200c82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:20.406201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe29eb-7aa4-4887-b558-a9465ee16dfd None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:20.411731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:20.432474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:20.444271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999feda6-7c5a-4bcf-825b-906cdf271913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:20.444988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999feda6-7c5a-4bcf-825b-906cdf271913 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:20.445362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999feda6-7c5a-4bcf-825b-906cdf271913 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:20.445736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999feda6-7c5a-4bcf-825b-906cdf271913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:20.494850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d3f24-31dd-449e-9428-eb41a4200c82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:20.496338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d3f24-31dd-449e-9428-eb41a4200c82 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:20.551759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999feda6-7c5a-4bcf-825b-906cdf271913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:20.553250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999feda6-7c5a-4bcf-825b-906cdf271913 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:20.693086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 179/371] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:20.693923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d3f24-31dd-449e-9428-eb41a4200c82 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:20.763417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 193/372] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:20.764537 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999feda6-7c5a-4bcf-825b-906cdf271913 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:20.821739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:20.834392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:20.904468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d3f24-31dd-449e-9428-eb41a4200c82 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:20.983840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999feda6-7c5a-4bcf-825b-906cdf271913 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:20.986051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790ead7-c3af-40fb-bffb-d4f06e6e1d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:20.991121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790ead7-c3af-40fb-bffb-d4f06e6e1d12 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:20.991121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790ead7-c3af-40fb-bffb-d4f06e6e1d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:20.991121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790ead7-c3af-40fb-bffb-d4f06e6e1d12 None admin] PATH_INFO: `/v3/users/6ce9882d360f4476a1b5852ab873e6cd` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:20.994742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790ead7-c3af-40fb-bffb-d4f06e6e1d12 None admin] RBAC: Authorizing `identity:get_user(user_id=6ce9882d360f4476a1b5852ab873e6cd)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:20.995642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790ead7-c3af-40fb-bffb-d4f06e6e1d12 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:20.997582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 180/373] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:09:20 2022] GET /identity/v3/users/6ce9882d360f4476a1b5852ab873e6cd => generated 293 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:20.998128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790ead7-c3af-40fb-bffb-d4f06e6e1d12 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:21.054922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899a22c-076d-4a8c-8f82-8b5a886f4f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:21.057059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899a22c-076d-4a8c-8f82-8b5a886f4f48 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:21.057648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899a22c-076d-4a8c-8f82-8b5a886f4f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:21.058741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899a22c-076d-4a8c-8f82-8b5a886f4f48 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:21.060816 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899a22c-076d-4a8c-8f82-8b5a886f4f48 None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:21.062619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899a22c-076d-4a8c-8f82-8b5a886f4f48 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:21.071566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 194/374] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:20 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:21.072813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899a22c-076d-4a8c-8f82-8b5a886f4f48 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:21.217291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790ead7-c3af-40fb-bffb-d4f06e6e1d12 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:21.252836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:21.267422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4914480-e9d9-468f-9d51-8e235878f4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:21.268425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4914480-e9d9-468f-9d51-8e235878f4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:21.268881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4914480-e9d9-468f-9d51-8e235878f4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:21.269346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4914480-e9d9-468f-9d51-8e235878f4c9 None admin] PATH_INFO: `/v3/projects/05bb534ab3954f32aad6d8b3751d3ac9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:21.274112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4914480-e9d9-468f-9d51-8e235878f4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=05bb534ab3954f32aad6d8b3751d3ac9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:21.274112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4914480-e9d9-468f-9d51-8e235878f4c9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:21.274327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899a22c-076d-4a8c-8f82-8b5a886f4f48 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:21.275473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 181/375] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:20 2022] GET /identity/v3/projects/05bb534ab3954f32aad6d8b3751d3ac9 => generated 344 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:21.280912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4914480-e9d9-468f-9d51-8e235878f4c9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:21.285765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:21.338082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6620885-bfc3-478b-8f77-194c1c857e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:21.338792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6620885-bfc3-478b-8f77-194c1c857e25 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:21.339097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6620885-bfc3-478b-8f77-194c1c857e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:21.339318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6620885-bfc3-478b-8f77-194c1c857e25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:21.340130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6620885-bfc3-478b-8f77-194c1c857e25 None admin] RBAC: Adding query filter params (role.id=8e94dcc6eec7482fab6c8aba670a4b6c, user.id=6ce9882d360f4476a1b5852ab873e6cd, scope.system=all) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:21.340442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6620885-bfc3-478b-8f77-194c1c857e25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:21.341923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6620885-bfc3-478b-8f77-194c1c857e25 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:21.359610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 195/376] 10.222.0.69 () {56 vars in 1308 bytes} [Thu Jul 28 06:09:21 2022] GET /identity/v3/role_assignments?user.id=6ce9882d360f4476a1b5852ab873e6cd&scope.system=all&role.id=8e94dcc6eec7482fab6c8aba670a4b6c => generated 510 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:21.360276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6620885-bfc3-478b-8f77-194c1c857e25 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:21.477420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4914480-e9d9-468f-9d51-8e235878f4c9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:21.519430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26084d54-e30c-42ee-9699-5c870138efed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:21.520172 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26084d54-e30c-42ee-9699-5c870138efed None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:21.520677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26084d54-e30c-42ee-9699-5c870138efed None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:21.520979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26084d54-e30c-42ee-9699-5c870138efed None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:21.522462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26084d54-e30c-42ee-9699-5c870138efed None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:21.523961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26084d54-e30c-42ee-9699-5c870138efed None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:21.526032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 182/377] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:21 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:21.527263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916042b9-bd5d-4a4c-9158-897ee85c6e24 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:21.527549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916042b9-bd5d-4a4c-9158-897ee85c6e24 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:21.527825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916042b9-bd5d-4a4c-9158-897ee85c6e24 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:21.529515 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 183/378] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:21.542847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156d675-0c38-4273-b9be-5e59eadca8d8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:21.543137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156d675-0c38-4273-b9be-5e59eadca8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:21.543427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156d675-0c38-4273-b9be-5e59eadca8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:21.576969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0156d675-0c38-4273-b9be-5e59eadca8d8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:21.576969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0156d675-0c38-4273-b9be-5e59eadca8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:21.582493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6620885-bfc3-478b-8f77-194c1c857e25 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:21.635028 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cfad9-dd45-4175-aa12-6738d5e9d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:21.636018 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cfad9-dd45-4175-aa12-6738d5e9d059 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:21.636573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cfad9-dd45-4175-aa12-6738d5e9d059 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:21.637036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cfad9-dd45-4175-aa12-6738d5e9d059 None admin] PATH_INFO: `/v3/groups/9c6c4224a52a46b98f2b89fd2401853f` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:21.648550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cfad9-dd45-4175-aa12-6738d5e9d059 None admin] RBAC: Authorizing `identity:get_group(group_id=9c6c4224a52a46b98f2b89fd2401853f)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:21.650061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cfad9-dd45-4175-aa12-6738d5e9d059 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:21.651922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 196/379] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:21 2022] GET /identity/v3/groups/9c6c4224a52a46b98f2b89fd2401853f => generated 229 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:21.654209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:21.657255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cfad9-dd45-4175-aa12-6738d5e9d059 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:21.661272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 184/380] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:21.667519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156d675-0c38-4273-b9be-5e59eadca8d8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:21.675352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:21.802865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cfad9-dd45-4175-aa12-6738d5e9d059 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:21.807158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156d675-0c38-4273-b9be-5e59eadca8d8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:21.844139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1440a4d-9d99-49fa-b6a1-802438a324ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:21.845370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1440a4d-9d99-49fa-b6a1-802438a324ff None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:21.845834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1440a4d-9d99-49fa-b6a1-802438a324ff None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:21.845958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ac3d9-c336-470e-be19-9515b263af3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:21.846418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1440a4d-9d99-49fa-b6a1-802438a324ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:21.846558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ac3d9-c336-470e-be19-9515b263af3d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:21.846803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ac3d9-c336-470e-be19-9515b263af3d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:21.847085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ac3d9-c336-470e-be19-9515b263af3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:21.848699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ac3d9-c336-470e-be19-9515b263af3d None admin] RBAC: Adding query filter params (group.id=9c6c4224a52a46b98f2b89fd2401853f, role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=05bb534ab3954f32aad6d8b3751d3ac9) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:21.848980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1440a4d-9d99-49fa-b6a1-802438a324ff None admin] RBAC: Adding query filter params (group.id=72ec25c95e614ac8b7558060c5e30674, role.id=c7b4bfc82f3b46cfaa10773dad7cd03a, scope.project.id=0660a31d921d44188bb944fcdbb9b027) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:21.849115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ac3d9-c336-470e-be19-9515b263af3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:21.849891 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1440a4d-9d99-49fa-b6a1-802438a324ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:21.851739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1440a4d-9d99-49fa-b6a1-802438a324ff None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:21.851983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ac3d9-c336-470e-be19-9515b263af3d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:21.858339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 197/381] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 28 06:09:21 2022] GET /identity/v3/role_assignments?group.id=72ec25c95e614ac8b7558060c5e30674&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=c7b4bfc82f3b46cfaa10773dad7cd03a => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:21.859403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1440a4d-9d99-49fa-b6a1-802438a324ff None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:21.865974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 185/382] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 28 06:09:21 2022] GET /identity/v3/role_assignments?group.id=9c6c4224a52a46b98f2b89fd2401853f&scope.project.id=05bb534ab3954f32aad6d8b3751d3ac9&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:21.956145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:21.982872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1440a4d-9d99-49fa-b6a1-802438a324ff None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:21.991544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77714b57-43d6-4d0d-8e82-bcf2bf200dbf None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:21.991924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77714b57-43d6-4d0d-8e82-bcf2bf200dbf None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:21.992284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77714b57-43d6-4d0d-8e82-bcf2bf200dbf None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:21.994139 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 186/383] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:22.002112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341762b7-15d3-448a-8ae8-406ea559b1f9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.002396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341762b7-15d3-448a-8ae8-406ea559b1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.002816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341762b7-15d3-448a-8ae8-406ea559b1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.026192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b586e9-a60a-4e38-bdf9-beadb3e6b464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:22.026912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b586e9-a60a-4e38-bdf9-beadb3e6b464 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.027203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b586e9-a60a-4e38-bdf9-beadb3e6b464 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.027482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b586e9-a60a-4e38-bdf9-beadb3e6b464 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.040043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b586e9-a60a-4e38-bdf9-beadb3e6b464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:22.042519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-341762b7-15d3-448a-8ae8-406ea559b1f9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:22.043751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-341762b7-15d3-448a-8ae8-406ea559b1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-06b586e9-a60a-4e38-bdf9-beadb3e6b464 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:22.046630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:22.047556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:22.048790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 198/384] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:09:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:22.054873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b586e9-a60a-4e38-bdf9-beadb3e6b464 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:22.167681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:22.173138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b586e9-a60a-4e38-bdf9-beadb3e6b464 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:22.186024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 187/385] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:22.211356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341762b7-15d3-448a-8ae8-406ea559b1f9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:22.213931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b39c65-7817-421e-a8fa-0cd526bb51e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:22.214565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39c65-7817-421e-a8fa-0cd526bb51e2 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.215693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39c65-7817-421e-a8fa-0cd526bb51e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.215693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39c65-7817-421e-a8fa-0cd526bb51e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.216064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b39c65-7817-421e-a8fa-0cd526bb51e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:22.216376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b39c65-7817-421e-a8fa-0cd526bb51e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:22.217802 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b39c65-7817-421e-a8fa-0cd526bb51e2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:22.220324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:22.228077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 199/386] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:22.233870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b39c65-7817-421e-a8fa-0cd526bb51e2 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:22.323371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341762b7-15d3-448a-8ae8-406ea559b1f9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:22.351431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b39c65-7817-421e-a8fa-0cd526bb51e2 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:22.366485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfab882-f075-4e78-bde0-76415123c29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:22.367504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfab882-f075-4e78-bde0-76415123c29b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.368036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfab882-f075-4e78-bde0-76415123c29b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.368516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfab882-f075-4e78-bde0-76415123c29b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.383656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfab882-f075-4e78-bde0-76415123c29b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-dbfab882-f075-4e78-bde0-76415123c29b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:22.389799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:22.390722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:22.391526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:22.394256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 188/387] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:22.396317 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a736014-b4c2-44d9-ab54-d0e5d1a8e77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:22.396975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a736014-b4c2-44d9-ab54-d0e5d1a8e77c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.397261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a736014-b4c2-44d9-ab54-d0e5d1a8e77c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.397604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a736014-b4c2-44d9-ab54-d0e5d1a8e77c None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.398782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a736014-b4c2-44d9-ab54-d0e5d1a8e77c None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:22.400457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfab882-f075-4e78-bde0-76415123c29b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:22.400585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a736014-b4c2-44d9-ab54-d0e5d1a8e77c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:22.402559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 200/388] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:22 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:22.408321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a736014-b4c2-44d9-ab54-d0e5d1a8e77c None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:22.479972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:22.482771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:22.532702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a736014-b4c2-44d9-ab54-d0e5d1a8e77c None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:22.551836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfab882-f075-4e78-bde0-76415123c29b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:22.581610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d679ee-ad3e-4b3f-bc8f-1f20143d62b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:22.582797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d679ee-ad3e-4b3f-bc8f-1f20143d62b4 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.583294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d679ee-ad3e-4b3f-bc8f-1f20143d62b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.583701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d679ee-ad3e-4b3f-bc8f-1f20143d62b4 None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.584923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d679ee-ad3e-4b3f-bc8f-1f20143d62b4 None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:22.586523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d679ee-ad3e-4b3f-bc8f-1f20143d62b4 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:22.588542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 201/389] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:22 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:22.589858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6d5d1-07ba-4a4d-a090-3f3280418e7e None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.590148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6d5d1-07ba-4a4d-a090-3f3280418e7e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.590425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6d5d1-07ba-4a4d-a090-3f3280418e7e None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.591634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 202/390] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:22.596551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6d5d1-07ba-4a4d-a090-3f3280418e7e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:22.598987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58d0bd-2103-4359-b8b3-bafe7185abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:22.599698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58d0bd-2103-4359-b8b3-bafe7185abed None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.600260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58d0bd-2103-4359-b8b3-bafe7185abed None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.600604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58d0bd-2103-4359-b8b3-bafe7185abed None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.602081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58d0bd-2103-4359-b8b3-bafe7185abed None admin] RBAC: Adding query filter params (name=member) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:22.602395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58d0bd-2103-4359-b8b3-bafe7185abed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:22.603796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58d0bd-2103-4359-b8b3-bafe7185abed None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:22.618013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 189/391] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:22.619652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9506-dd86-4917-8ec8-5ef77af334a5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.620160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9506-dd86-4917-8ec8-5ef77af334a5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.620635 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9506-dd86-4917-8ec8-5ef77af334a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.645802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-265d9506-dd86-4917-8ec8-5ef77af334a5 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:22.647354 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-265d9506-dd86-4917-8ec8-5ef77af334a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:22.715307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6d5d1-07ba-4a4d-a090-3f3280418e7e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:22.720885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 190/392] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:22.722101 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d9506-dd86-4917-8ec8-5ef77af334a5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:22.771750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:22.773744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac196f6-40aa-49a6-b192-9b9b68e83e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:22.774433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac196f6-40aa-49a6-b192-9b9b68e83e86 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.774717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac196f6-40aa-49a6-b192-9b9b68e83e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.775065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac196f6-40aa-49a6-b192-9b9b68e83e86 None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/groups/72ec25c95e614ac8b7558060c5e30674/roles/c7b4bfc82f3b46cfaa10773dad7cd03a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.777495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac196f6-40aa-49a6-b192-9b9b68e83e86 None admin] RBAC: Authorizing `identity:create_grant(project_id=e646eb9882194d2f8e2bf2ec959ecff3, group_id=72ec25c95e614ac8b7558060c5e30674, role_id=c7b4bfc82f3b46cfaa10773dad7cd03a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:22.778948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac196f6-40aa-49a6-b192-9b9b68e83e86 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:22.781524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:22.830383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d9506-dd86-4917-8ec8-5ef77af334a5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:22.865727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 203/393] 10.222.0.69 () {58 vars in 1442 bytes} [Thu Jul 28 06:09:22 2022] PUT /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/groups/72ec25c95e614ac8b7558060c5e30674/roles/c7b4bfc82f3b46cfaa10773dad7cd03a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:22.866806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac196f6-40aa-49a6-b192-9b9b68e83e86 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:22.882543 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377e6cf-02db-4260-aec2-e399657ff3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:22.885172 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377e6cf-02db-4260-aec2-e399657ff3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:22.885901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377e6cf-02db-4260-aec2-e399657ff3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:22.886445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377e6cf-02db-4260-aec2-e399657ff3c6 None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:22.888201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377e6cf-02db-4260-aec2-e399657ff3c6 None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:22.890436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377e6cf-02db-4260-aec2-e399657ff3c6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:22.894109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 191/394] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:22 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:22.900364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377e6cf-02db-4260-aec2-e399657ff3c6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:23.057516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac196f6-40aa-49a6-b192-9b9b68e83e86 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:23.095804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a872bc-bb33-4018-b293-511953962f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:23.096532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a872bc-bb33-4018-b293-511953962f0f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:23.096969 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a872bc-bb33-4018-b293-511953962f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:23.097096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a872bc-bb33-4018-b293-511953962f0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:23.104187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377e6cf-02db-4260-aec2-e399657ff3c6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:23.108771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a872bc-bb33-4018-b293-511953962f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-41a872bc-bb33-4018-b293-511953962f0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:23.114956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:23.115941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:23.116807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 204/395] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:23.126037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a872bc-bb33-4018-b293-511953962f0f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:23.129954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:23.156752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67458f94-2318-43c0-ba5a-184842033782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:23.157846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67458f94-2318-43c0-ba5a-184842033782 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:23.158439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67458f94-2318-43c0-ba5a-184842033782 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:23.158923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67458f94-2318-43c0-ba5a-184842033782 None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:23.162060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67458f94-2318-43c0-ba5a-184842033782 None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:23.163820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67458f94-2318-43c0-ba5a-184842033782 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:23.166030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 192/396] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:22 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:23.174424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67458f94-2318-43c0-ba5a-184842033782 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:23.207715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:23.284411 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a872bc-bb33-4018-b293-511953962f0f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:23.324658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a68bae-8ab5-4113-ad41-cf7f366997d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:23.325542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a68bae-8ab5-4113-ad41-cf7f366997d2 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:23.325832 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a68bae-8ab5-4113-ad41-cf7f366997d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:23.326115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a68bae-8ab5-4113-ad41-cf7f366997d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:23.327928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a68bae-8ab5-4113-ad41-cf7f366997d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:23.328297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a68bae-8ab5-4113-ad41-cf7f366997d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:23.330758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a68bae-8ab5-4113-ad41-cf7f366997d2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:23.351530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 205/397] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:23.359842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a68bae-8ab5-4113-ad41-cf7f366997d2 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:23.421980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67458f94-2318-43c0-ba5a-184842033782 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:23.476190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a7718-538a-4ef1-b758-935b8437ba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:23.477419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a7718-538a-4ef1-b758-935b8437ba9a None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:23.477909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a7718-538a-4ef1-b758-935b8437ba9a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:23.478392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a7718-538a-4ef1-b758-935b8437ba9a None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/groups/72ec25c95e614ac8b7558060c5e30674/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:23.480756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a7718-538a-4ef1-b758-935b8437ba9a None admin] RBAC: Authorizing `identity:create_grant(project_id=e646eb9882194d2f8e2bf2ec959ecff3, group_id=72ec25c95e614ac8b7558060c5e30674, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:23.482599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a7718-538a-4ef1-b758-935b8437ba9a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:23.567026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a68bae-8ab5-4113-ad41-cf7f366997d2 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:23.598973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:23.605133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 193/398] 10.222.0.69 () {58 vars in 1442 bytes} [Thu Jul 28 06:09:23 2022] PUT /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3/groups/72ec25c95e614ac8b7558060c5e30674/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:23.641925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ce9e8-2433-4628-912f-9555d3596254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:23.642964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ce9e8-2433-4628-912f-9555d3596254 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:23.643474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ce9e8-2433-4628-912f-9555d3596254 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:23.643959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ce9e8-2433-4628-912f-9555d3596254 None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:23.645668 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ce9e8-2433-4628-912f-9555d3596254 None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:23.647512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ce9e8-2433-4628-912f-9555d3596254 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:23.650033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 206/399] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:23 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:23.655841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a7718-538a-4ef1-b758-935b8437ba9a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:23.772422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:23.857463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a7718-538a-4ef1-b758-935b8437ba9a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:23.904090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21378203-526c-47a7-a53d-756643c4f8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:23.904757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21378203-526c-47a7-a53d-756643c4f8bd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:23.905044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21378203-526c-47a7-a53d-756643c4f8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:23.905366 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21378203-526c-47a7-a53d-756643c4f8bd None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:23.906567 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21378203-526c-47a7-a53d-756643c4f8bd None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:23.908026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21378203-526c-47a7-a53d-756643c4f8bd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:23.910616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 194/400] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:23 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:23.916608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ce9e8-2433-4628-912f-9555d3596254 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:24.065799 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ce9e8-2433-4628-912f-9555d3596254 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:24.116143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ea963-14cd-48de-b401-2e7c89cbd8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:24.117295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ea963-14cd-48de-b401-2e7c89cbd8d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:24.117801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ea963-14cd-48de-b401-2e7c89cbd8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:24.118351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ea963-14cd-48de-b401-2e7c89cbd8d2 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027/groups/72ec25c95e614ac8b7558060c5e30674/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:24.120602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ea963-14cd-48de-b401-2e7c89cbd8d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0660a31d921d44188bb944fcdbb9b027, group_id=72ec25c95e614ac8b7558060c5e30674, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:24.122312 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ea963-14cd-48de-b401-2e7c89cbd8d2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:24.243696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 207/401] 10.222.0.69 () {58 vars in 1442 bytes} [Thu Jul 28 06:09:23 2022] PUT /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027/groups/72ec25c95e614ac8b7558060c5e30674/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:24.382747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30090ff7-2ac1-4d20-8ba6-df05d8faa924 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:24.383433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30090ff7-2ac1-4d20-8ba6-df05d8faa924 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:24.383930 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30090ff7-2ac1-4d20-8ba6-df05d8faa924 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:24.386066 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 195/402] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:24.392482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f784ee-71e8-4947-bd28-6528d64fa769 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:24.393116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f784ee-71e8-4947-bd28-6528d64fa769 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:24.393682 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f784ee-71e8-4947-bd28-6528d64fa769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:24.419371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-c4f784ee-71e8-4947-bd28-6528d64fa769 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:24.420970 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c4f784ee-71e8-4947-bd28-6528d64fa769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:24.468436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:24.499797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 208/403] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:24.524141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30090ff7-2ac1-4d20-8ba6-df05d8faa924 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:24.678278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30090ff7-2ac1-4d20-8ba6-df05d8faa924 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:24.690609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:24.735044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a6e0c-f41f-4f3a-9c09-77b203c3e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:24.735723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a6e0c-f41f-4f3a-9c09-77b203c3e641 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:24.736006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a6e0c-f41f-4f3a-9c09-77b203c3e641 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:24.736286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a6e0c-f41f-4f3a-9c09-77b203c3e641 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:24.748331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a6e0c-f41f-4f3a-9c09-77b203c3e641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-896a6e0c-f41f-4f3a-9c09-77b203c3e641 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:24.753374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:24.754877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:24.755807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 196/404] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:09:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:24.762922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f784ee-71e8-4947-bd28-6528d64fa769 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:24.886749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f784ee-71e8-4947-bd28-6528d64fa769 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:24.932956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08bd37-88ea-495c-b569-d0c040eff61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:24.933999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08bd37-88ea-495c-b569-d0c040eff61e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:24.934524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08bd37-88ea-495c-b569-d0c040eff61e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:24.935011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08bd37-88ea-495c-b569-d0c040eff61e None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:24.935974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08bd37-88ea-495c-b569-d0c040eff61e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:24.936543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08bd37-88ea-495c-b569-d0c040eff61e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:24.938320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08bd37-88ea-495c-b569-d0c040eff61e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:24.952172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 209/405] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:24.958843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a6e0c-f41f-4f3a-9c09-77b203c3e641 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:24.999457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb4d80-ff37-44b4-a54d-a01d2ae952b7 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:25.000092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb4d80-ff37-44b4-a54d-a01d2ae952b7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:25.000585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb4d80-ff37-44b4-a54d-a01d2ae952b7 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:25.002495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 210/406] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:09:25.008617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860e17d-78d9-4b83-ab28-b9ace0acf6c6 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:25.009181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860e17d-78d9-4b83-ab28-b9ace0acf6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:25.009724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860e17d-78d9-4b83-ab28-b9ace0acf6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:25.029476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f860e17d-78d9-4b83-ab28-b9ace0acf6c6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:25.031267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f860e17d-78d9-4b83-ab28-b9ace0acf6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:25.070201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:25.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a6e0c-f41f-4f3a-9c09-77b203c3e641 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:25.162667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 211/407] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:25.177969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72a56d-6348-45b3-804a-82c61ca12d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:25.178734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72a56d-6348-45b3-804a-82c61ca12d58 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:25.179096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72a56d-6348-45b3-804a-82c61ca12d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:25.179450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72a56d-6348-45b3-804a-82c61ca12d58 None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:25.180760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72a56d-6348-45b3-804a-82c61ca12d58 None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:25.182326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72a56d-6348-45b3-804a-82c61ca12d58 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:25.184518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 197/408] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:24 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:25.191957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860e17d-78d9-4b83-ab28-b9ace0acf6c6 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:25.193586 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72a56d-6348-45b3-804a-82c61ca12d58 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:25.256420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:25.319247 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860e17d-78d9-4b83-ab28-b9ace0acf6c6 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:25.345186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72a56d-6348-45b3-804a-82c61ca12d58 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:25.365010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27d66a-09a1-44fa-a165-e235b2ce065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:25.365010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27d66a-09a1-44fa-a165-e235b2ce065a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:25.365010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27d66a-09a1-44fa-a165-e235b2ce065a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:25.365010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27d66a-09a1-44fa-a165-e235b2ce065a None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:25.368477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27d66a-09a1-44fa-a165-e235b2ce065a None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:25.370150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27d66a-09a1-44fa-a165-e235b2ce065a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:25.372290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 212/409] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:25 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:25.373720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662ec5b-cf84-4294-9ed0-46750d50017c None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:25.374155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662ec5b-cf84-4294-9ed0-46750d50017c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:25.374622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662ec5b-cf84-4294-9ed0-46750d50017c None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:25.376247 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 213/410] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:25.377422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662ec5b-cf84-4294-9ed0-46750d50017c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:25.392609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464e733-0bd9-4bbe-9906-61de30d58678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:25.393485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e733-0bd9-4bbe-9906-61de30d58678 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:25.393868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e733-0bd9-4bbe-9906-61de30d58678 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:25.394277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e733-0bd9-4bbe-9906-61de30d58678 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:25.414966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464e733-0bd9-4bbe-9906-61de30d58678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:25.422591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7464e733-0bd9-4bbe-9906-61de30d58678 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:25.425135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:25.426765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:25.427591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:25.431839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 198/411] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:09:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:25.433235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5c844-6614-428d-a415-79e74c7804c7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:25.434086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5c844-6614-428d-a415-79e74c7804c7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:25.434375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5c844-6614-428d-a415-79e74c7804c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:25.458638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9fe5c844-6614-428d-a415-79e74c7804c7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:25.460071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9fe5c844-6614-428d-a415-79e74c7804c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:25.501792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662ec5b-cf84-4294-9ed0-46750d50017c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:25.548295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff7b03-68a0-407d-ae6d-c407245038ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:25.549257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff7b03-68a0-407d-ae6d-c407245038ce None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:25.549572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff7b03-68a0-407d-ae6d-c407245038ce None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:25.549852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff7b03-68a0-407d-ae6d-c407245038ce None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027/groups/72ec25c95e614ac8b7558060c5e30674/roles/c7b4bfc82f3b46cfaa10773dad7cd03a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:25.552936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff7b03-68a0-407d-ae6d-c407245038ce None admin] RBAC: Authorizing `identity:create_grant(project_id=0660a31d921d44188bb944fcdbb9b027, group_id=72ec25c95e614ac8b7558060c5e30674, role_id=c7b4bfc82f3b46cfaa10773dad7cd03a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:25.554381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 199/412] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:25.555876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5c844-6614-428d-a415-79e74c7804c7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:25.561494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff7b03-68a0-407d-ae6d-c407245038ce None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:25.591881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:25.640960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 214/413] 10.222.0.69 () {58 vars in 1442 bytes} [Thu Jul 28 06:09:25 2022] PUT /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027/groups/72ec25c95e614ac8b7558060c5e30674/roles/c7b4bfc82f3b46cfaa10773dad7cd03a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:25.640960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff7b03-68a0-407d-ae6d-c407245038ce None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:25.728499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5c844-6614-428d-a415-79e74c7804c7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:25.813999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e3f02-4529-49a0-bfa5-17b03c8a1db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:25.814651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3f02-4529-49a0-bfa5-17b03c8a1db9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:25.814933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3f02-4529-49a0-bfa5-17b03c8a1db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:25.815211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3f02-4529-49a0-bfa5-17b03c8a1db9 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:25.815956 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e3f02-4529-49a0-bfa5-17b03c8a1db9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:25.816446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e3f02-4529-49a0-bfa5-17b03c8a1db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:25.818230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e3f02-4529-49a0-bfa5-17b03c8a1db9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:25.824716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:25.837238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 200/414] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:25.839421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff7b03-68a0-407d-ae6d-c407245038ce None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:25.848734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e3f02-4529-49a0-bfa5-17b03c8a1db9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:25.884372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c628f-20e3-4e89-b31d-e92e35893ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:25.885618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c628f-20e3-4e89-b31d-e92e35893ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:25.886154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c628f-20e3-4e89-b31d-e92e35893ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:25.886782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c628f-20e3-4e89-b31d-e92e35893ab9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:25.904751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c628f-20e3-4e89-b31d-e92e35893ab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-587c628f-20e3-4e89-b31d-e92e35893ab9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:25.911297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:25.912185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:25.912978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:25.920355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 215/415] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:09:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:25.921846 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c628f-20e3-4e89-b31d-e92e35893ab9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:25.964975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:25.976282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e3f02-4529-49a0-bfa5-17b03c8a1db9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:26.042887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9c6cd-5885-4d52-95e4-b43c6000e9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:26.044003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9c6cd-5885-4d52-95e4-b43c6000e9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.045555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9c6cd-5885-4d52-95e4-b43c6000e9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.046152 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9c6cd-5885-4d52-95e4-b43c6000e9c8 None admin] PATH_INFO: `/v3/groups/9c6c4224a52a46b98f2b89fd2401853f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.048840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9c6cd-5885-4d52-95e4-b43c6000e9c8 None admin] RBAC: Authorizing `identity:get_group(group_id=9c6c4224a52a46b98f2b89fd2401853f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:26.051251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9c6cd-5885-4d52-95e4-b43c6000e9c8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:26.056291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 201/416] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:25 2022] GET /identity/v3/groups/9c6c4224a52a46b98f2b89fd2401853f => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:26.064046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9c6cd-5885-4d52-95e4-b43c6000e9c8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:26.115277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c628f-20e3-4e89-b31d-e92e35893ab9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:26.157474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26a6ab-f5ba-4a14-bb41-d00de36edfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:26.158439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26a6ab-f5ba-4a14-bb41-d00de36edfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.158875 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26a6ab-f5ba-4a14-bb41-d00de36edfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.159868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26a6ab-f5ba-4a14-bb41-d00de36edfe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.161868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26a6ab-f5ba-4a14-bb41-d00de36edfe3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:26.162337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26a6ab-f5ba-4a14-bb41-d00de36edfe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:26.164844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26a6ab-f5ba-4a14-bb41-d00de36edfe3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:26.181367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 216/417] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:26.181367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de200d20-3102-4915-b2ad-f1de99f70089 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.181367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de200d20-3102-4915-b2ad-f1de99f70089 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.181367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de200d20-3102-4915-b2ad-f1de99f70089 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.184313 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 217/418] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:26.185580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab556d-9645-41eb-87cc-6170a5a8a68c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.185876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab556d-9645-41eb-87cc-6170a5a8a68c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.186161 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab556d-9645-41eb-87cc-6170a5a8a68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.205364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2bab556d-9645-41eb-87cc-6170a5a8a68c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:26.206617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2bab556d-9645-41eb-87cc-6170a5a8a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:26.218703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:26.237525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9c6cd-5885-4d52-95e4-b43c6000e9c8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:26.279716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f354e0-6416-4299-abee-e98a5dd9048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:26.280876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f354e0-6416-4299-abee-e98a5dd9048d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.281331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f354e0-6416-4299-abee-e98a5dd9048d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.282192 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f354e0-6416-4299-abee-e98a5dd9048d None admin] PATH_INFO: `/v3/projects/05bb534ab3954f32aad6d8b3751d3ac9` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.283803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f354e0-6416-4299-abee-e98a5dd9048d None admin] RBAC: Authorizing `identity:get_project(project_id=05bb534ab3954f32aad6d8b3751d3ac9)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:26.286349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f354e0-6416-4299-abee-e98a5dd9048d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:26.289487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 202/419] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:26 2022] GET /identity/v3/projects/05bb534ab3954f32aad6d8b3751d3ac9 => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:26.290568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f354e0-6416-4299-abee-e98a5dd9048d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:26.297922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 218/420] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:26.298795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab556d-9645-41eb-87cc-6170a5a8a68c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:26.324311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:26.428739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f354e0-6416-4299-abee-e98a5dd9048d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:26.440422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab556d-9645-41eb-87cc-6170a5a8a68c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:26.477268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe9245-05de-475f-a650-a5372af14bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:26.477970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe9245-05de-475f-a650-a5372af14bae None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.478258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe9245-05de-475f-a650-a5372af14bae None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.478537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe9245-05de-475f-a650-a5372af14bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.482848 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b1087-8069-4a14-8e8d-f0c564c19c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:26.484098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b1087-8069-4a14-8e8d-f0c564c19c7c None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.484098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b1087-8069-4a14-8e8d-f0c564c19c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.484098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b1087-8069-4a14-8e8d-f0c564c19c7c None admin] PATH_INFO: `/v3/projects/05bb534ab3954f32aad6d8b3751d3ac9/groups/9c6c4224a52a46b98f2b89fd2401853f/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.486141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b1087-8069-4a14-8e8d-f0c564c19c7c None admin] RBAC: Authorizing `identity:create_grant(project_id=05bb534ab3954f32aad6d8b3751d3ac9, group_id=9c6c4224a52a46b98f2b89fd2401853f, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:26.487620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b1087-8069-4a14-8e8d-f0c564c19c7c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:26.559915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe9245-05de-475f-a650-a5372af14bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:26.569341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe9245-05de-475f-a650-a5372af14bae None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:26.569768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 219/421] 10.222.0.69 () {58 vars in 1442 bytes} [Thu Jul 28 06:09:26 2022] PUT /identity/v3/projects/05bb534ab3954f32aad6d8b3751d3ac9/groups/9c6c4224a52a46b98f2b89fd2401853f/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:26.571182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b1087-8069-4a14-8e8d-f0c564c19c7c None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:26.589339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:26.600951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:26.698478 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 203/422] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:26.708048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe9245-05de-475f-a650-a5372af14bae None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:26.804337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b1087-8069-4a14-8e8d-f0c564c19c7c None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:26.844966 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c67ac-2053-457e-99d6-5d6320bc4dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:26.845559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c67ac-2053-457e-99d6-5d6320bc4dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.845847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c67ac-2053-457e-99d6-5d6320bc4dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.846125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c67ac-2053-457e-99d6-5d6320bc4dc0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.847029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe9245-05de-475f-a650-a5372af14bae None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:26.857996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c67ac-2053-457e-99d6-5d6320bc4dc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-5a1c67ac-2053-457e-99d6-5d6320bc4dc0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:26.864012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:26.864902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:26.865830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:26.869140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 220/423] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:26.870568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a68b-9080-4678-bc77-403e1abbd7f0 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.870854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a68b-9080-4678-bc77-403e1abbd7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.871334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a68b-9080-4678-bc77-403e1abbd7f0 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.872869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 221/424] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:26.877645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a68b-9080-4678-bc77-403e1abbd7f0 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:26.884019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:26.893801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2ce8f-e0b7-47f1-b1d4-bdbfc68a4c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:26.896188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2ce8f-e0b7-47f1-b1d4-bdbfc68a4c2e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.896188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2ce8f-e0b7-47f1-b1d4-bdbfc68a4c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.896188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2ce8f-e0b7-47f1-b1d4-bdbfc68a4c2e None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.897571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2ce8f-e0b7-47f1-b1d4-bdbfc68a4c2e None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:26.899384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2ce8f-e0b7-47f1-b1d4-bdbfc68a4c2e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:26.901717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 204/425] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:26 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:26.903082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f48ddd-80f9-4fe9-b81b-228921d81e8e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:26.903539 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f48ddd-80f9-4fe9-b81b-228921d81e8e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:26.903985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f48ddd-80f9-4fe9-b81b-228921d81e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:26.924144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-49f48ddd-80f9-4fe9-b81b-228921d81e8e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:26.925990 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-49f48ddd-80f9-4fe9-b81b-228921d81e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:26.953686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:26.998626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a68b-9080-4678-bc77-403e1abbd7f0 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:27.000290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 205/426] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:27.001648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f48ddd-80f9-4fe9-b81b-228921d81e8e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:27.051113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018be0c-971a-4fbc-a30b-d49fa7cd75b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:27.051695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018be0c-971a-4fbc-a30b-d49fa7cd75b7 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:27.051982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018be0c-971a-4fbc-a30b-d49fa7cd75b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:27.052257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018be0c-971a-4fbc-a30b-d49fa7cd75b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:27.052991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018be0c-971a-4fbc-a30b-d49fa7cd75b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:27.053339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018be0c-971a-4fbc-a30b-d49fa7cd75b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:27.055476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018be0c-971a-4fbc-a30b-d49fa7cd75b7 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:27.067559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 222/427] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:27.068420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018be0c-971a-4fbc-a30b-d49fa7cd75b7 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:27.120345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f48ddd-80f9-4fe9-b81b-228921d81e8e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:27.147133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:27.158705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cf76d-8f84-4d45-a6a6-1e59426b20ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:27.159724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cf76d-8f84-4d45-a6a6-1e59426b20ec None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:27.160781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cf76d-8f84-4d45-a6a6-1e59426b20ec None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:27.161326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cf76d-8f84-4d45-a6a6-1e59426b20ec None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:27.162709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cf76d-8f84-4d45-a6a6-1e59426b20ec None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:27.164401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cf76d-8f84-4d45-a6a6-1e59426b20ec None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:27.166546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 206/428] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:27 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:27.167609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cf76d-8f84-4d45-a6a6-1e59426b20ec None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:27.186926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018be0c-971a-4fbc-a30b-d49fa7cd75b7 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:27.215941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:27.225577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a92453-a184-4290-86b2-7403a57b2249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:27.228042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a92453-a184-4290-86b2-7403a57b2249 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:27.228042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a92453-a184-4290-86b2-7403a57b2249 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:27.228042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a92453-a184-4290-86b2-7403a57b2249 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:27.238786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a92453-a184-4290-86b2-7403a57b2249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-a8a92453-a184-4290-86b2-7403a57b2249 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:27.244042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:27.244910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:09:27.245958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:27.248946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 223/429] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:27.249964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a92453-a184-4290-86b2-7403a57b2249 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:27.281842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cf76d-8f84-4d45-a6a6-1e59426b20ec None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:27.320426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68223b46-d61d-4c4d-bea2-1974d0a9d479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:27.321209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68223b46-d61d-4c4d-bea2-1974d0a9d479 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:27.321615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68223b46-d61d-4c4d-bea2-1974d0a9d479 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:27.322032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68223b46-d61d-4c4d-bea2-1974d0a9d479 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:27.361850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a92453-a184-4290-86b2-7403a57b2249 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:27.390036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:27.401203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecc8c3-d382-4900-b558-a2ed27bbf703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:27.401546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68223b46-d61d-4c4d-bea2-1974d0a9d479 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:27.402053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecc8c3-d382-4900-b558-a2ed27bbf703 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:27.402404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecc8c3-d382-4900-b558-a2ed27bbf703 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:27.411021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecc8c3-d382-4900-b558-a2ed27bbf703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:27.411167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68223b46-d61d-4c4d-bea2-1974d0a9d479 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:27.411245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecc8c3-d382-4900-b558-a2ed27bbf703 None admin] RBAC: Adding query filter params (group.id=72ec25c95e614ac8b7558060c5e30674, role.id=c7b4bfc82f3b46cfaa10773dad7cd03a, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:27.411245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecc8c3-d382-4900-b558-a2ed27bbf703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:27.411245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecc8c3-d382-4900-b558-a2ed27bbf703 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:27.413056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 224/430] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 28 06:09:27 2022] GET /identity/v3/role_assignments?group.id=72ec25c95e614ac8b7558060c5e30674&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=c7b4bfc82f3b46cfaa10773dad7cd03a => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:27.413657 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecc8c3-d382-4900-b558-a2ed27bbf703 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:27.475919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:27.518337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 207/431] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:27.530500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68223b46-d61d-4c4d-bea2-1974d0a9d479 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:27.538327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecc8c3-d382-4900-b558-a2ed27bbf703 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:27.578478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10a158-e19e-4c47-a6fe-cbcc57520c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:27.579123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10a158-e19e-4c47-a6fe-cbcc57520c37 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:27.579416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10a158-e19e-4c47-a6fe-cbcc57520c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:27.579686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10a158-e19e-4c47-a6fe-cbcc57520c37 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:27.580446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10a158-e19e-4c47-a6fe-cbcc57520c37 None admin] RBAC: Adding query filter params (name=member) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:27.580842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10a158-e19e-4c47-a6fe-cbcc57520c37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:27.582867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10a158-e19e-4c47-a6fe-cbcc57520c37 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:27.592854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 225/432] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:09:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:27.599360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10a158-e19e-4c47-a6fe-cbcc57520c37 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:27.636626 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:27.651510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68223b46-d61d-4c4d-bea2-1974d0a9d479 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:27.696210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d644ad-12fe-4ff8-b423-2c0081ec9a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:27.697745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d644ad-12fe-4ff8-b423-2c0081ec9a5f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:27.698566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d644ad-12fe-4ff8-b423-2c0081ec9a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:27.699354 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d644ad-12fe-4ff8-b423-2c0081ec9a5f None admin] PATH_INFO: `/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:27.701252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d644ad-12fe-4ff8-b423-2c0081ec9a5f None admin] RBAC: Authorizing `identity:get_project(project_id=e646eb9882194d2f8e2bf2ec959ecff3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:27.702967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d644ad-12fe-4ff8-b423-2c0081ec9a5f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:27.705203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 208/433] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:27 2022] GET /identity/v3/projects/e646eb9882194d2f8e2bf2ec959ecff3 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:27.711222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d644ad-12fe-4ff8-b423-2c0081ec9a5f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:27.736440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10a158-e19e-4c47-a6fe-cbcc57520c37 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:27.750944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:27.773932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6e192-16d0-4ef5-9744-b063289910a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:27.774908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6e192-16d0-4ef5-9744-b063289910a3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:27.775400 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6e192-16d0-4ef5-9744-b063289910a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:27.775904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6e192-16d0-4ef5-9744-b063289910a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:27.829342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d644ad-12fe-4ff8-b423-2c0081ec9a5f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:27.852817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6e192-16d0-4ef5-9744-b063289910a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:27.854379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6e192-16d0-4ef5-9744-b063289910a3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:27.881444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8676e6-666f-4754-aa86-cb04af607607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:27.882469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8676e6-666f-4754-aa86-cb04af607607 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:27.882984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8676e6-666f-4754-aa86-cb04af607607 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:27.884131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8676e6-666f-4754-aa86-cb04af607607 None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:27.885855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8676e6-666f-4754-aa86-cb04af607607 None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:27.888539 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8676e6-666f-4754-aa86-cb04af607607 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:27.891769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 209/434] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:27 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:27.898255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8676e6-666f-4754-aa86-cb04af607607 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:27.931982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:27.995785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 226/435] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:27.997869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d28493-e989-4ef9-8566-8ec98fb32a69 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:27.998354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d28493-e989-4ef9-8566-8ec98fb32a69 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:27.998787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d28493-e989-4ef9-8566-8ec98fb32a69 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.001011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 227/436] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:28.002533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d28493-e989-4ef9-8566-8ec98fb32a69 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:28.021028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8676e6-666f-4754-aa86-cb04af607607 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:28.026151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:28.058388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f5dfd-a90a-4e3c-93f1-2ec1ff8545f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:28.059016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f5dfd-a90a-4e3c-93f1-2ec1ff8545f9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:28.059307 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f5dfd-a90a-4e3c-93f1-2ec1ff8545f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:28.059583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f5dfd-a90a-4e3c-93f1-2ec1ff8545f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.060379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f5dfd-a90a-4e3c-93f1-2ec1ff8545f9 None admin] RBAC: Adding query filter params (group.id=72ec25c95e614ac8b7558060c5e30674, role.id=c4fc73849d664b2588bd2de7372aa839, scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:28.060696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f5dfd-a90a-4e3c-93f1-2ec1ff8545f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:28.062190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f5dfd-a90a-4e3c-93f1-2ec1ff8545f9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:28.069077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 210/437] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 28 06:09:27 2022] GET /identity/v3/role_assignments?group.id=72ec25c95e614ac8b7558060c5e30674&scope.project.id=e646eb9882194d2f8e2bf2ec959ecff3&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:28.070392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df930a17-4516-4059-8738-7b1028290c0b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:28.070718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df930a17-4516-4059-8738-7b1028290c0b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:28.070966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df930a17-4516-4059-8738-7b1028290c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.093489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-df930a17-4516-4059-8738-7b1028290c0b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:28.094793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-df930a17-4516-4059-8738-7b1028290c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:28.157807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d28493-e989-4ef9-8566-8ec98fb32a69 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:28.174822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 211/438] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:28.198568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df930a17-4516-4059-8738-7b1028290c0b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:28.198898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91405e-9c0b-49ea-bbda-138c47259cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:28.200099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91405e-9c0b-49ea-bbda-138c47259cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:28.200685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91405e-9c0b-49ea-bbda-138c47259cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:28.201147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91405e-9c0b-49ea-bbda-138c47259cb0 None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.202727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91405e-9c0b-49ea-bbda-138c47259cb0 None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:28.204428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91405e-9c0b-49ea-bbda-138c47259cb0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:28.206889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 228/439] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:28 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:28.214965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91405e-9c0b-49ea-bbda-138c47259cb0 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:28.225778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:28.310767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:28.326564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df930a17-4516-4059-8738-7b1028290c0b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:28.337482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91405e-9c0b-49ea-bbda-138c47259cb0 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:28.368999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96594e64-bd30-44a5-b8d2-8f7c964bc2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:28.370372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96594e64-bd30-44a5-b8d2-8f7c964bc2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:28.370372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96594e64-bd30-44a5-b8d2-8f7c964bc2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:28.370564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96594e64-bd30-44a5-b8d2-8f7c964bc2c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.391168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96594e64-bd30-44a5-b8d2-8f7c964bc2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:28.392176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523743b-cac2-4ab1-8456-9c9ef9842469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:28.392853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523743b-cac2-4ab1-8456-9c9ef9842469 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:28.393145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523743b-cac2-4ab1-8456-9c9ef9842469 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:28.393456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523743b-cac2-4ab1-8456-9c9ef9842469 None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.395312 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523743b-cac2-4ab1-8456-9c9ef9842469 None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:28.396959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523743b-cac2-4ab1-8456-9c9ef9842469 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:28.398935 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 229/440] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:28 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-96594e64-bd30-44a5-b8d2-8f7c964bc2c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:28.399132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:28.400258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:28.401491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 212/441] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:09:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:28.404947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523743b-cac2-4ab1-8456-9c9ef9842469 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:28.412204 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96594e64-bd30-44a5-b8d2-8f7c964bc2c2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:28.521592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:28.549675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523743b-cac2-4ab1-8456-9c9ef9842469 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:28.549933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96594e64-bd30-44a5-b8d2-8f7c964bc2c2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:28.593142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34985960-804f-4568-82bb-7305a9caa80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:28.594191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34985960-804f-4568-82bb-7305a9caa80a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:28.594677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34985960-804f-4568-82bb-7305a9caa80a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:28.595141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6967d03-ace3-48d3-82cc-53acab00ed0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:28.595364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34985960-804f-4568-82bb-7305a9caa80a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.595779 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6967d03-ace3-48d3-82cc-53acab00ed0c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:28.596082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6967d03-ace3-48d3-82cc-53acab00ed0c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:28.596352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6967d03-ace3-48d3-82cc-53acab00ed0c None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.596937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34985960-804f-4568-82bb-7305a9caa80a None admin] RBAC: Adding query filter params (group.id=72ec25c95e614ac8b7558060c5e30674, role.id=c4fc73849d664b2588bd2de7372aa839, scope.project.id=0660a31d921d44188bb944fcdbb9b027) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:28.597099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6967d03-ace3-48d3-82cc-53acab00ed0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:28.598058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6967d03-ace3-48d3-82cc-53acab00ed0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:28.598479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34985960-804f-4568-82bb-7305a9caa80a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:28.599516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6967d03-ace3-48d3-82cc-53acab00ed0c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:28.600441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34985960-804f-4568-82bb-7305a9caa80a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:28.609136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 230/442] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 28 06:09:28 2022] GET /identity/v3/role_assignments?group.id=72ec25c95e614ac8b7558060c5e30674&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=c4fc73849d664b2588bd2de7372aa839 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:28.610401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 213/443] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:28.616765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34985960-804f-4568-82bb-7305a9caa80a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:28.637681 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:28.744553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34985960-804f-4568-82bb-7305a9caa80a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:28.786288 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7746ce6-6b45-48bd-93ff-2376c7c7d550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:28.786885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7746ce6-6b45-48bd-93ff-2376c7c7d550 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:28.787126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7746ce6-6b45-48bd-93ff-2376c7c7d550 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:28.787409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7746ce6-6b45-48bd-93ff-2376c7c7d550 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.865126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7746ce6-6b45-48bd-93ff-2376c7c7d550 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:28.866717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7746ce6-6b45-48bd-93ff-2376c7c7d550 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:28.884979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:28.975619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709cd2d-42af-4d35-bfaa-97ff1bfee5b4 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:28.976134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709cd2d-42af-4d35-bfaa-97ff1bfee5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:28.976223 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709cd2d-42af-4d35-bfaa-97ff1bfee5b4 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.977668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 214/444] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:09:28.983050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09506fb2-8a6e-4a06-a364-1cba5dcd3270 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:28.983342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09506fb2-8a6e-4a06-a364-1cba5dcd3270 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:28.983783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09506fb2-8a6e-4a06-a364-1cba5dcd3270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:28.984308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 231/445] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:28.990729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7746ce6-6b45-48bd-93ff-2376c7c7d550 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:29.008358 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-09506fb2-8a6e-4a06-a364-1cba5dcd3270 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:29.009591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-09506fb2-8a6e-4a06-a364-1cba5dcd3270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:29.081704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 215/446] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:29.101096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7746ce6-6b45-48bd-93ff-2376c7c7d550 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:29.104987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09506fb2-8a6e-4a06-a364-1cba5dcd3270 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:29.116053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:29.136234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f1e63-7ddf-40d1-a0a7-85ca9d80107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:29.136881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f1e63-7ddf-40d1-a0a7-85ca9d80107a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:29.137169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f1e63-7ddf-40d1-a0a7-85ca9d80107a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:29.137491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f1e63-7ddf-40d1-a0a7-85ca9d80107a None admin] PATH_INFO: `/v3/projects/0660a31d921d44188bb944fcdbb9b027` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:29.138969 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f1e63-7ddf-40d1-a0a7-85ca9d80107a None admin] RBAC: Authorizing `identity:get_project(project_id=0660a31d921d44188bb944fcdbb9b027)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:29.140571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f1e63-7ddf-40d1-a0a7-85ca9d80107a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:29.142261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 232/447] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:28 2022] GET /identity/v3/projects/0660a31d921d44188bb944fcdbb9b027 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:29.147854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f1e63-7ddf-40d1-a0a7-85ca9d80107a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:29.183397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:29.207611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09506fb2-8a6e-4a06-a364-1cba5dcd3270 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:29.243992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b43efa-c64b-4955-bf4a-1107e834790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:29.244645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b43efa-c64b-4955-bf4a-1107e834790a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:29.244935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b43efa-c64b-4955-bf4a-1107e834790a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:29.245217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b43efa-c64b-4955-bf4a-1107e834790a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:29.248701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f1e63-7ddf-40d1-a0a7-85ca9d80107a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:29.256261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b43efa-c64b-4955-bf4a-1107e834790a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a9b43efa-c64b-4955-bf4a-1107e834790a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:29.261338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:29.262408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:29.263521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 216/448] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:09:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:29.268633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b43efa-c64b-4955-bf4a-1107e834790a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:29.289972 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16c333-8f5d-4c1c-92ff-1dfaf0b28a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:29.290622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16c333-8f5d-4c1c-92ff-1dfaf0b28a97 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:29.290911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16c333-8f5d-4c1c-92ff-1dfaf0b28a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:29.291196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16c333-8f5d-4c1c-92ff-1dfaf0b28a97 None admin] PATH_INFO: `/v3/groups/72ec25c95e614ac8b7558060c5e30674` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:29.292366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16c333-8f5d-4c1c-92ff-1dfaf0b28a97 None admin] RBAC: Authorizing `identity:get_group(group_id=72ec25c95e614ac8b7558060c5e30674)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:29.293850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16c333-8f5d-4c1c-92ff-1dfaf0b28a97 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:29.295452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 233/449] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:29 2022] GET /identity/v3/groups/72ec25c95e614ac8b7558060c5e30674 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:29.301264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16c333-8f5d-4c1c-92ff-1dfaf0b28a97 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:29.353832 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:29.370568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b43efa-c64b-4955-bf4a-1107e834790a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:29.406717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16c333-8f5d-4c1c-92ff-1dfaf0b28a97 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:29.420674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79fb75-42b2-4d1b-b72d-4f2c5b97adc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:29.421325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79fb75-42b2-4d1b-b72d-4f2c5b97adc6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:29.421617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79fb75-42b2-4d1b-b72d-4f2c5b97adc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:29.421895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79fb75-42b2-4d1b-b72d-4f2c5b97adc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:29.422635 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79fb75-42b2-4d1b-b72d-4f2c5b97adc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:29.423295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79fb75-42b2-4d1b-b72d-4f2c5b97adc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:29.425000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79fb75-42b2-4d1b-b72d-4f2c5b97adc6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:29.426682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:29.435187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 217/450] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:29.440857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79fb75-42b2-4d1b-b72d-4f2c5b97adc6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:29.452868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd99527-5acb-4338-8480-1bdc9faeed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:29.453517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd99527-5acb-4338-8480-1bdc9faeed86 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:29.453803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd99527-5acb-4338-8480-1bdc9faeed86 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:29.454081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd99527-5acb-4338-8480-1bdc9faeed86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:29.454877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd99527-5acb-4338-8480-1bdc9faeed86 None admin] RBAC: Adding query filter params (group.id=72ec25c95e614ac8b7558060c5e30674, role.id=c7b4bfc82f3b46cfaa10773dad7cd03a, scope.project.id=0660a31d921d44188bb944fcdbb9b027) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:29.455186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd99527-5acb-4338-8480-1bdc9faeed86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:29.456685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd99527-5acb-4338-8480-1bdc9faeed86 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:29.463240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 234/451] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 28 06:09:29 2022] GET /identity/v3/role_assignments?group.id=72ec25c95e614ac8b7558060c5e30674&scope.project.id=0660a31d921d44188bb944fcdbb9b027&role.id=c7b4bfc82f3b46cfaa10773dad7cd03a => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:29.551461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79fb75-42b2-4d1b-b72d-4f2c5b97adc6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:29.588719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633e00c-db6e-4c70-80a7-d965c6093bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:29.589382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633e00c-db6e-4c70-80a7-d965c6093bbf None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:29.589676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633e00c-db6e-4c70-80a7-d965c6093bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:29.589960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633e00c-db6e-4c70-80a7-d965c6093bbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:29.657234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:29.663532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633e00c-db6e-4c70-80a7-d965c6093bbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:29.665023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633e00c-db6e-4c70-80a7-d965c6093bbf None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:29.793282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 218/452] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:09:29.794679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd99527-5acb-4338-8480-1bdc9faeed86 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:29.896659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd99527-5acb-4338-8480-1bdc9faeed86 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:29.911446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:29.931237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686d00b-f6fc-4fd1-8bf1-c26db4f256e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:29.931881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686d00b-f6fc-4fd1-8bf1-c26db4f256e1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:29.932170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686d00b-f6fc-4fd1-8bf1-c26db4f256e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:29.932452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686d00b-f6fc-4fd1-8bf1-c26db4f256e1 None admin] PATH_INFO: `/v3/projects/05bb534ab3954f32aad6d8b3751d3ac9` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:29.933653 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686d00b-f6fc-4fd1-8bf1-c26db4f256e1 None admin] RBAC: Authorizing `identity:get_project(project_id=05bb534ab3954f32aad6d8b3751d3ac9)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:29.935115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686d00b-f6fc-4fd1-8bf1-c26db4f256e1 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:29.936803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 235/453] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:09:29 2022] GET /identity/v3/projects/05bb534ab3954f32aad6d8b3751d3ac9 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:29.942535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633e00c-db6e-4c70-80a7-d965c6093bbf None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:30.032282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:30.038610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633e00c-db6e-4c70-80a7-d965c6093bbf None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:30.072443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fd57b-6a91-4870-9626-22e8019031fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:30.073090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fd57b-6a91-4870-9626-22e8019031fd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:30.073410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fd57b-6a91-4870-9626-22e8019031fd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:30.073696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fd57b-6a91-4870-9626-22e8019031fd None admin] PATH_INFO: `/v3/groups/9c6c4224a52a46b98f2b89fd2401853f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:30.074907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fd57b-6a91-4870-9626-22e8019031fd None admin] RBAC: Authorizing `identity:get_group(group_id=9c6c4224a52a46b98f2b89fd2401853f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:30.076379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fd57b-6a91-4870-9626-22e8019031fd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:30.078014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 219/454] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 28 06:09:29 2022] GET /identity/v3/groups/9c6c4224a52a46b98f2b89fd2401853f => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:30.084104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686d00b-f6fc-4fd1-8bf1-c26db4f256e1 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:30.191964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686d00b-f6fc-4fd1-8bf1-c26db4f256e1 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:30.224435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540365d9-3f59-4e32-9851-4540f15a897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:30.225054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540365d9-3f59-4e32-9851-4540f15a897f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:30.225364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540365d9-3f59-4e32-9851-4540f15a897f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:30.225651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540365d9-3f59-4e32-9851-4540f15a897f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:30.226442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540365d9-3f59-4e32-9851-4540f15a897f None admin] RBAC: Adding query filter params (group.id=9c6c4224a52a46b98f2b89fd2401853f, role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=05bb534ab3954f32aad6d8b3751d3ac9) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:30.226762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540365d9-3f59-4e32-9851-4540f15a897f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:30.228194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540365d9-3f59-4e32-9851-4540f15a897f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:30.233863 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 236/455] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 28 06:09:30 2022] GET /identity/v3/role_assignments?group.id=9c6c4224a52a46b98f2b89fd2401853f&scope.project.id=05bb534ab3954f32aad6d8b3751d3ac9&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:32.770878 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9dfd1-db49-4f1e-af31-c013b11fdb2e None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:32.771894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9dfd1-db49-4f1e-af31-c013b11fdb2e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:32.772374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9dfd1-db49-4f1e-af31-c013b11fdb2e None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:32.774319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 220/456] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:32.790918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63b453-258e-429e-ab31-0a6ca868a395 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:32.791833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63b453-258e-429e-ab31-0a6ca868a395 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:32.792326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63b453-258e-429e-ab31-0a6ca868a395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:32.822392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4b63b453-258e-429e-ab31-0a6ca868a395 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:32.823959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4b63b453-258e-429e-ab31-0a6ca868a395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:32.881574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:32.923184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 237/457] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:32.973278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9dfd1-db49-4f1e-af31-c013b11fdb2e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:33.066947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd568f-4c9f-4423-a5cc-1ac0f296ff2d None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.067690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd568f-4c9f-4423-a5cc-1ac0f296ff2d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.068207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd568f-4c9f-4423-a5cc-1ac0f296ff2d None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.070173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 238/458] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:09:33.075759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd162790-4e95-49c3-b585-b283324aec55 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.076311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd162790-4e95-49c3-b585-b283324aec55 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.076786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd162790-4e95-49c3-b585-b283324aec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.096055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-cd162790-4e95-49c3-b585-b283324aec55 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:33.098221 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-cd162790-4e95-49c3-b585-b283324aec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:33.126393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9dfd1-db49-4f1e-af31-c013b11fdb2e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:33.174278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 239/459] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:33.176771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2a305-0a09-4b2a-a084-b0ecc5b45683 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.177687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2a305-0a09-4b2a-a084-b0ecc5b45683 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.178569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2a305-0a09-4b2a-a084-b0ecc5b45683 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.180507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 240/460] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:33.182821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd15bb-657f-4ab7-a53f-de4be8729f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:33.183625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd15bb-657f-4ab7-a53f-de4be8729f34 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.184052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd15bb-657f-4ab7-a53f-de4be8729f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.184450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd15bb-657f-4ab7-a53f-de4be8729f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.192788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a575d0-e6c2-400d-88d7-8b0584c6e37e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.193280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a575d0-e6c2-400d-88d7-8b0584c6e37e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.193721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a575d0-e6c2-400d-88d7-8b0584c6e37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.208066 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd15bb-657f-4ab7-a53f-de4be8729f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:33.210009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd15bb-657f-4ab7-a53f-de4be8729f34 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:33.224385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e1a575d0-e6c2-400d-88d7-8b0584c6e37e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:33.225936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e1a575d0-e6c2-400d-88d7-8b0584c6e37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e8dd15bb-657f-4ab7-a53f-de4be8729f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:33.230624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:33.231627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:33.232472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:33.232472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:33.232472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:33.232472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:33.232741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 221/461] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:33.233958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd15bb-657f-4ab7-a53f-de4be8729f34 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:33.260572 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:33.326893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 241/462] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:33.327969 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a575d0-e6c2-400d-88d7-8b0584c6e37e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:33.368589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd15bb-657f-4ab7-a53f-de4be8729f34 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:33.384714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:33.409945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c314f-9c2c-4a6d-8b3b-f3cb07b96a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:33.410593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c314f-9c2c-4a6d-8b3b-f3cb07b96a8a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.410877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c314f-9c2c-4a6d-8b3b-f3cb07b96a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.411158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c314f-9c2c-4a6d-8b3b-f3cb07b96a8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.422607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c314f-9c2c-4a6d-8b3b-f3cb07b96a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:33.424075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c314f-9c2c-4a6d-8b3b-f3cb07b96a8a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-9b0c314f-9c2c-4a6d-8b3b-f3cb07b96a8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:33.439553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:33.440617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:33.442285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:33.442285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:33.442285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:33.442285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:33.443766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 222/463] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:33.445097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d154d-6f61-4c90-9ee3-adaf807b6ef4 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.445406 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d154d-6f61-4c90-9ee3-adaf807b6ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.445696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d154d-6f61-4c90-9ee3-adaf807b6ef4 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.446945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 223/464] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:33.447775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d154d-6f61-4c90-9ee3-adaf807b6ef4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:33.453470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a575d0-e6c2-400d-88d7-8b0584c6e37e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:33.490887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb51b05-e0b3-4fdd-a97c-2bc1b16a1acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:33.491515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb51b05-e0b3-4fdd-a97c-2bc1b16a1acb None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.491808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb51b05-e0b3-4fdd-a97c-2bc1b16a1acb None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.492085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb51b05-e0b3-4fdd-a97c-2bc1b16a1acb None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.492945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb51b05-e0b3-4fdd-a97c-2bc1b16a1acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:33.493256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb51b05-e0b3-4fdd-a97c-2bc1b16a1acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:33.494664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb51b05-e0b3-4fdd-a97c-2bc1b16a1acb None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:33.511171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 242/465] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:33.511171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb51b05-e0b3-4fdd-a97c-2bc1b16a1acb None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:33.528225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:33.566529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d154d-6f61-4c90-9ee3-adaf807b6ef4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:33.605664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd0032-a24a-4515-84e0-79315927d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:33.606241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd0032-a24a-4515-84e0-79315927d843 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.606526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd0032-a24a-4515-84e0-79315927d843 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.606833 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd0032-a24a-4515-84e0-79315927d843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.608201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:33.618723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb51b05-e0b3-4fdd-a97c-2bc1b16a1acb None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:33.621087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd0032-a24a-4515-84e0-79315927d843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:33.621342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd0032-a24a-4515-84e0-79315927d843 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-73bd0032-a24a-4515-84e0-79315927d843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:33.636874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:33.637978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:33.639305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:33.639305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:33.639305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:33.639305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:33.640974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 224/466] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:33.642342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aebe96-0ad4-481a-be0c-2391c41a3d25 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.642625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aebe96-0ad4-481a-be0c-2391c41a3d25 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.642902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aebe96-0ad4-481a-be0c-2391c41a3d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.662576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a7aebe96-0ad4-481a-be0c-2391c41a3d25 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:33.663494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473f2d-5b33-43a4-b3d2-e6b777de4b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:33.663857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a7aebe96-0ad4-481a-be0c-2391c41a3d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:33.664121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473f2d-5b33-43a4-b3d2-e6b777de4b23 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.664410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473f2d-5b33-43a4-b3d2-e6b777de4b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.664682 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473f2d-5b33-43a4-b3d2-e6b777de4b23 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.665402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473f2d-5b33-43a4-b3d2-e6b777de4b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:33.665712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473f2d-5b33-43a4-b3d2-e6b777de4b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:33.667090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473f2d-5b33-43a4-b3d2-e6b777de4b23 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:33.680406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 243/467] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:33.681590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473f2d-5b33-43a4-b3d2-e6b777de4b23 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:33.743107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 225/468] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:33.744242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aebe96-0ad4-481a-be0c-2391c41a3d25 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:33.799345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473f2d-5b33-43a4-b3d2-e6b777de4b23 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:33.806303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:33.839667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585d82a-d432-45da-b3f9-e6faff177b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:33.840490 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585d82a-d432-45da-b3f9-e6faff177b04 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.840771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585d82a-d432-45da-b3f9-e6faff177b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.841363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585d82a-d432-45da-b3f9-e6faff177b04 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.842061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585d82a-d432-45da-b3f9-e6faff177b04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:33.843502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585d82a-d432-45da-b3f9-e6faff177b04 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:33.858588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aebe96-0ad4-481a-be0c-2391c41a3d25 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:33.899361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2ef73-fed6-4807-a328-b9a935f1006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:33.899989 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2ef73-fed6-4807-a328-b9a935f1006c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:33.900276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2ef73-fed6-4807-a328-b9a935f1006c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:33.900553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2ef73-fed6-4807-a328-b9a935f1006c None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:33.901254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2ef73-fed6-4807-a328-b9a935f1006c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:33.901594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2ef73-fed6-4807-a328-b9a935f1006c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:33.903221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2ef73-fed6-4807-a328-b9a935f1006c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:33.917720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 226/469] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:33.918606 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2ef73-fed6-4807-a328-b9a935f1006c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:33.961345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 244/470] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:09:33 2022] POST /identity/v3/users => generated 246 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:09:33.961345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585d82a-d432-45da-b3f9-e6faff177b04 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:33.977092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:34.079873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2ef73-fed6-4807-a328-b9a935f1006c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:34.086323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585d82a-d432-45da-b3f9-e6faff177b04 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:34.095059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:34.125433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429483d6-b3d1-47d9-a8ee-abae28149081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:34.126397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429483d6-b3d1-47d9-a8ee-abae28149081 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:34.126884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429483d6-b3d1-47d9-a8ee-abae28149081 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:34.127370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429483d6-b3d1-47d9-a8ee-abae28149081 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:34.128396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429483d6-b3d1-47d9-a8ee-abae28149081 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:34.130113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429483d6-b3d1-47d9-a8ee-abae28149081 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:34.145838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf440b5-b020-4ead-b67a-35182f751df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:34.145838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf440b5-b020-4ead-b67a-35182f751df1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:34.145838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf440b5-b020-4ead-b67a-35182f751df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:34.145838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf440b5-b020-4ead-b67a-35182f751df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:34.156427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf440b5-b020-4ead-b67a-35182f751df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:34.157972 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf440b5-b020-4ead-b67a-35182f751df1 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-9cf440b5-b020-4ead-b67a-35182f751df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:34.174148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:34.175118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:34.176449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:34.176449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:34.176449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:34.176449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:34.176449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 245/471] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:34.176449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf440b5-b020-4ead-b67a-35182f751df1 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:34.207732 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 227/472] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:09:33 2022] POST /identity/v3/users => generated 248 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:09:34.209588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64a9da-6352-4286-8cc4-07fe15a01353 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:34.210112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64a9da-6352-4286-8cc4-07fe15a01353 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:34.210597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64a9da-6352-4286-8cc4-07fe15a01353 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:34.216625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 228/473] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:34.218064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64a9da-6352-4286-8cc4-07fe15a01353 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:34.276717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:34.325808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf440b5-b020-4ead-b67a-35182f751df1 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:34.352142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64a9da-6352-4286-8cc4-07fe15a01353 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:34.368834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba445f9-2894-4c07-bb34-067e90b0ceaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:34.368834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba445f9-2894-4c07-bb34-067e90b0ceaf None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:34.368834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba445f9-2894-4c07-bb34-067e90b0ceaf None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:34.369170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba445f9-2894-4c07-bb34-067e90b0ceaf None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:34.370053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba445f9-2894-4c07-bb34-067e90b0ceaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:34.371280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba445f9-2894-4c07-bb34-067e90b0ceaf None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:34.397262 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10239c-d3e7-4806-abdb-8494bf89ae3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:34.397952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10239c-d3e7-4806-abdb-8494bf89ae3f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:34.398271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10239c-d3e7-4806-abdb-8494bf89ae3f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:34.398682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10239c-d3e7-4806-abdb-8494bf89ae3f None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:34.399238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a10239c-d3e7-4806-abdb-8494bf89ae3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:34.399546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a10239c-d3e7-4806-abdb-8494bf89ae3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:34.401018 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a10239c-d3e7-4806-abdb-8494bf89ae3f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:34.416424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 229/474] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:34.418688 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303668ee-876c-4c94-89c2-9bcd61930ec1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:34.418688 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303668ee-876c-4c94-89c2-9bcd61930ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:34.418688 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303668ee-876c-4c94-89c2-9bcd61930ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:34.437164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-303668ee-876c-4c94-89c2-9bcd61930ec1 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:34.438542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-303668ee-876c-4c94-89c2-9bcd61930ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:34.462381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:34.488627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 246/475] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:09:34 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:09:34.492474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba445f9-2894-4c07-bb34-067e90b0ceaf None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:34.556324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 230/476] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:34.584072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303668ee-876c-4c94-89c2-9bcd61930ec1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:34.690230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba445f9-2894-4c07-bb34-067e90b0ceaf None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:34.716203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:34.732101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb124908-d053-4c83-a533-6e49e51c81b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:34.733000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124908-d053-4c83-a533-6e49e51c81b8 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:34.733497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124908-d053-4c83-a533-6e49e51c81b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:34.733924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124908-d053-4c83-a533-6e49e51c81b8 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:34.734765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb124908-d053-4c83-a533-6e49e51c81b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:34.736447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb124908-d053-4c83-a533-6e49e51c81b8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:34.751847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303668ee-876c-4c94-89c2-9bcd61930ec1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:34.791408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43c8a8-53c7-4623-98c6-caf689db1cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:34.792072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43c8a8-53c7-4623-98c6-caf689db1cb0 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:34.792352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43c8a8-53c7-4623-98c6-caf689db1cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:34.792686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43c8a8-53c7-4623-98c6-caf689db1cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:34.793468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43c8a8-53c7-4623-98c6-caf689db1cb0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:34.794941 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43c8a8-53c7-4623-98c6-caf689db1cb0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:34.826911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 247/477] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:09:34 2022] POST /identity/v3/users => generated 249 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:09:34.849008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.sql.core [None req-aa43c8a8-53c7-4623-98c6-caf689db1cb0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 06:09:34.849008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 06:09:34.849008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [parameters: {'id': '70d5c5b7e7a648ffb807a1750877352a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 06:09:34.849008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80318) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-aa43c8a8-53c7-4623-98c6-caf689db1cb0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result.read() Jul 28 06:09:34.870900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:09:34.872010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application yield resource Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise self.value Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application session.commit() Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 06:09:34.873205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.session.flush() Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application compat.raise_( Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 06:09:34.874421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:09:34.875904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result.read() Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:09:34.882357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [parameters: {'id': '70d5c5b7e7a648ffb807a1750877352a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:34.884045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 06:09:34.885838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:34.887584 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 231/478] 10.222.0.69 () {60 vars in 1131 bytes} [Thu Jul 28 06:09:34 2022] POST /identity/v3/roles => generated 144 bytes in 289 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:34.887713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb124908-d053-4c83-a533-6e49e51c81b8 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:35.040484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb124908-d053-4c83-a533-6e49e51c81b8 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:35.077676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6d09a-7170-424d-add6-113bd9d5d088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:35.078344 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6d09a-7170-424d-add6-113bd9d5d088 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:35.078638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6d09a-7170-424d-add6-113bd9d5d088 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:35.078924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6d09a-7170-424d-add6-113bd9d5d088 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:35.091730 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6d09a-7170-424d-add6-113bd9d5d088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-94a6d09a-7170-424d-add6-113bd9d5d088 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:35.097367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:35.099634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:35.101438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 248/479] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:09:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:35.104398 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43c8a8-53c7-4623-98c6-caf689db1cb0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:35.148658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:35.250951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43c8a8-53c7-4623-98c6-caf689db1cb0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:35.288653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca0ea2-6969-4cba-92cf-12c65a1c5971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:35.289285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca0ea2-6969-4cba-92cf-12c65a1c5971 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:35.289597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca0ea2-6969-4cba-92cf-12c65a1c5971 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:35.289874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca0ea2-6969-4cba-92cf-12c65a1c5971 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:35.290661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca0ea2-6969-4cba-92cf-12c65a1c5971 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:35.290912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca0ea2-6969-4cba-92cf-12c65a1c5971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:35.292373 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca0ea2-6969-4cba-92cf-12c65a1c5971 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:35.301769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 232/480] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:36.318291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19747404-2954-4400-8983-da369e842716 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:36.318291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19747404-2954-4400-8983-da369e842716 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:36.318291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19747404-2954-4400-8983-da369e842716 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:36.318291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 249/481] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:36.335663 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa2246-1165-4c27-b323-eb7e839c3e6b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:36.336184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa2246-1165-4c27-b323-eb7e839c3e6b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:36.336297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa2246-1165-4c27-b323-eb7e839c3e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:36.374593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-72aa2246-1165-4c27-b323-eb7e839c3e6b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:36.375780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-72aa2246-1165-4c27-b323-eb7e839c3e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:36.464771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 233/482] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:36.504228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19747404-2954-4400-8983-da369e842716 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:36.573164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:36.660628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19747404-2954-4400-8983-da369e842716 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:36.711149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bc499-ddb4-48c4-b719-28f4dc3f5f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:36.711811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bc499-ddb4-48c4-b719-28f4dc3f5f9c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:36.712103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bc499-ddb4-48c4-b719-28f4dc3f5f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:36.712389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bc499-ddb4-48c4-b719-28f4dc3f5f9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:36.724069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bc499-ddb4-48c4-b719-28f4dc3f5f9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-da4bc499-ddb4-48c4-b719-28f4dc3f5f9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:36.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:36.731153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:36.732801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 250/483] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:36.740026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa2246-1165-4c27-b323-eb7e839c3e6b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:36.764360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b41fb-3027-4e1e-aae4-d96f8af36c56 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:36.764661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b41fb-3027-4e1e-aae4-d96f8af36c56 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:36.764930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b41fb-3027-4e1e-aae4-d96f8af36c56 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:36.767164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 251/484] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:09:36.771409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:36.774633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6bce0-7967-460c-8c43-866d57d997c3 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:36.774924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6bce0-7967-460c-8c43-866d57d997c3 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:36.775212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6bce0-7967-460c-8c43-866d57d997c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:36.811789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-34f6bce0-7967-460c-8c43-866d57d997c3 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:36.813019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-34f6bce0-7967-460c-8c43-866d57d997c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:36.903783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa2246-1165-4c27-b323-eb7e839c3e6b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:36.942936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b0a2b-c7c4-4e19-be08-96f6dc9506d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:36.943643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0a2b-c7c4-4e19-be08-96f6dc9506d3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:36.943923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0a2b-c7c4-4e19-be08-96f6dc9506d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:36.944193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0a2b-c7c4-4e19-be08-96f6dc9506d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:36.944977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b0a2b-c7c4-4e19-be08-96f6dc9506d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:36.945295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b0a2b-c7c4-4e19-be08-96f6dc9506d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:36.946972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b0a2b-c7c4-4e19-be08-96f6dc9506d3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:36.958665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 234/485] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:36.970815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b0a2b-c7c4-4e19-be08-96f6dc9506d3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:36.996040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 252/486] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:36.999829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c7346-451c-4e0b-94bc-0d06e3b0eead None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.000127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c7346-451c-4e0b-94bc-0d06e3b0eead None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.000408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c7346-451c-4e0b-94bc-0d06e3b0eead None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.002070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 253/487] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:37.008782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d514cc4-72a3-4d9e-ae14-bd008c3d0320 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.009112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d514cc4-72a3-4d9e-ae14-bd008c3d0320 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.009385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d514cc4-72a3-4d9e-ae14-bd008c3d0320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.037245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-1d514cc4-72a3-4d9e-ae14-bd008c3d0320 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:37.039048 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1d514cc4-72a3-4d9e-ae14-bd008c3d0320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:37.065463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:37.068318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:37.093509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b0a2b-c7c4-4e19-be08-96f6dc9506d3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:37.130025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 254/488] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:37.131162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d514cc4-72a3-4d9e-ae14-bd008c3d0320 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:37.142762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64456f72-d99e-40f5-908c-f8152b851f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:37.143430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64456f72-d99e-40f5-908c-f8152b851f95 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.143760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64456f72-d99e-40f5-908c-f8152b851f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.144210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64456f72-d99e-40f5-908c-f8152b851f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.165984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64456f72-d99e-40f5-908c-f8152b851f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:37.167417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64456f72-d99e-40f5-908c-f8152b851f95 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-64456f72-d99e-40f5-908c-f8152b851f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:37.185991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:37.186957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:37.187810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:37.187810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:37.187810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:37.187810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:37.192716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 235/489] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:37.193674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64456f72-d99e-40f5-908c-f8152b851f95 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:37.266252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d514cc4-72a3-4d9e-ae14-bd008c3d0320 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:37.304780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64456f72-d99e-40f5-908c-f8152b851f95 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:37.307478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05b3c3-0a40-48ca-95bc-497737b3c75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:37.308382 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05b3c3-0a40-48ca-95bc-497737b3c75c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.308972 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05b3c3-0a40-48ca-95bc-497737b3c75c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.309921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05b3c3-0a40-48ca-95bc-497737b3c75c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.325441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05b3c3-0a40-48ca-95bc-497737b3c75c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-3b05b3c3-0a40-48ca-95bc-497737b3c75c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:37.331692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:37.332564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:37.333359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:37.334741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:37.341417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 255/490] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:37.342418 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f26d9-a5ce-4634-bec2-b88fa719b88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:37.343161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f26d9-a5ce-4634-bec2-b88fa719b88b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.343361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f26d9-a5ce-4634-bec2-b88fa719b88b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.343612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05b3c3-0a40-48ca-95bc-497737b3c75c None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:37.348987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f26d9-a5ce-4634-bec2-b88fa719b88b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.357435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f26d9-a5ce-4634-bec2-b88fa719b88b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-293f26d9-a5ce-4634-bec2-b88fa719b88b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:37.363201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:37.364284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:37.365079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 236/491] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:37.370141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f26d9-a5ce-4634-bec2-b88fa719b88b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:37.383040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:37.470372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05b3c3-0a40-48ca-95bc-497737b3c75c None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:37.477686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f26d9-a5ce-4634-bec2-b88fa719b88b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:37.508099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916560b1-7bf0-4189-b596-c6ae0a44ef36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:37.509134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916560b1-7bf0-4189-b596-c6ae0a44ef36 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.509657 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916560b1-7bf0-4189-b596-c6ae0a44ef36 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.510132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916560b1-7bf0-4189-b596-c6ae0a44ef36 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.511300 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916560b1-7bf0-4189-b596-c6ae0a44ef36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:37.511838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916560b1-7bf0-4189-b596-c6ae0a44ef36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:37.513760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916560b1-7bf0-4189-b596-c6ae0a44ef36 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:37.515049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477afe8-efd9-4c11-84a1-f15a0975a2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:37.515684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477afe8-efd9-4c11-84a1-f15a0975a2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.515972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477afe8-efd9-4c11-84a1-f15a0975a2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.516449 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477afe8-efd9-4c11-84a1-f15a0975a2d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.517181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477afe8-efd9-4c11-84a1-f15a0975a2d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:37.518052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477afe8-efd9-4c11-84a1-f15a0975a2d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:37.520269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477afe8-efd9-4c11-84a1-f15a0975a2d5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:37.530572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 256/492] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:37.531750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916560b1-7bf0-4189-b596-c6ae0a44ef36 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:37.534592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 237/493] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:37.538049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477afe8-efd9-4c11-84a1-f15a0975a2d5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:37.595231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:37.644678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:37.654151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477afe8-efd9-4c11-84a1-f15a0975a2d5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:37.677608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916560b1-7bf0-4189-b596-c6ae0a44ef36 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:37.694360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854a412-de1a-4f46-94ea-559974093640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:37.694998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854a412-de1a-4f46-94ea-559974093640 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.695279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854a412-de1a-4f46-94ea-559974093640 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.695574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854a412-de1a-4f46-94ea-559974093640 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.703302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854a412-de1a-4f46-94ea-559974093640 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:37.704753 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854a412-de1a-4f46-94ea-559974093640 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:37.714681 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8322b9-2bdd-4092-bee7-8bbbcbf284d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:37.715315 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8322b9-2bdd-4092-bee7-8bbbcbf284d8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.715601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8322b9-2bdd-4092-bee7-8bbbcbf284d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.715881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8322b9-2bdd-4092-bee7-8bbbcbf284d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.716604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8322b9-2bdd-4092-bee7-8bbbcbf284d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:37.716913 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8322b9-2bdd-4092-bee7-8bbbcbf284d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:37.718321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8322b9-2bdd-4092-bee7-8bbbcbf284d8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c854a412-de1a-4f46-94ea-559974093640 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:37.718957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:37.719849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:37.720789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 238/494] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 28 06:09:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:37.721395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854a412-de1a-4f46-94ea-559974093640 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:37.733328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 257/495] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:37.739502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8360e-1cf0-4ec0-a2c6-949ec2453891 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.740219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8360e-1cf0-4ec0-a2c6-949ec2453891 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.740515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8360e-1cf0-4ec0-a2c6-949ec2453891 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.742368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 258/496] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:37.746769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8360e-1cf0-4ec0-a2c6-949ec2453891 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:37.835793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854a412-de1a-4f46-94ea-559974093640 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:37.845176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:37.860678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8360e-1cf0-4ec0-a2c6-949ec2453891 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:37.872198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c57fd-fad0-4c2d-a0bd-c4e819fab861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:37.872850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c57fd-fad0-4c2d-a0bd-c4e819fab861 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.873140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c57fd-fad0-4c2d-a0bd-c4e819fab861 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.873454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c57fd-fad0-4c2d-a0bd-c4e819fab861 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.884893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c57fd-fad0-4c2d-a0bd-c4e819fab861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:37.886853 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c57fd-fad0-4c2d-a0bd-c4e819fab861 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-1c0c57fd-fad0-4c2d-a0bd-c4e819fab861 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:37.903046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:37.904043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:37.904891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:37.904891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:37.904891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:37.904891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:37.905133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fca48-0a55-4f4b-8d0d-d80caf254a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:37.905133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fca48-0a55-4f4b-8d0d-d80caf254a27 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.905133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fca48-0a55-4f4b-8d0d-d80caf254a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.905355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 239/497] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:37.905355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c57fd-fad0-4c2d-a0bd-c4e819fab861 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:37.908924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fca48-0a55-4f4b-8d0d-d80caf254a27 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.910413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fca48-0a55-4f4b-8d0d-d80caf254a27 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:37.910748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fca48-0a55-4f4b-8d0d-d80caf254a27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:37.912875 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fca48-0a55-4f4b-8d0d-d80caf254a27 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:37.931279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:37.956698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 259/498] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 28 06:09:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:37.958111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52340d-2b6f-4f3d-b29d-7a01f77b8663 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:37.958407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52340d-2b6f-4f3d-b29d-7a01f77b8663 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:37.958715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52340d-2b6f-4f3d-b29d-7a01f77b8663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:37.978675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-6c52340d-2b6f-4f3d-b29d-7a01f77b8663 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:37.981096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6c52340d-2b6f-4f3d-b29d-7a01f77b8663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:38.016130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c57fd-fad0-4c2d-a0bd-c4e819fab861 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:38.047666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 260/499] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:38.048796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52340d-2b6f-4f3d-b29d-7a01f77b8663 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:38.051402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23793796-cdf6-4a13-b939-ac09992e3a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:38.052039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793796-cdf6-4a13-b939-ac09992e3a36 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.052319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793796-cdf6-4a13-b939-ac09992e3a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.052596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793796-cdf6-4a13-b939-ac09992e3a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.063136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23793796-cdf6-4a13-b939-ac09992e3a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:38.064598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23793796-cdf6-4a13-b939-ac09992e3a36 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-23793796-cdf6-4a13-b939-ac09992e3a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:38.079284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:38.080186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:38.081038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:38.081038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:38.081038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:38.081038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:38.081038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 240/500] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:38.081966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23793796-cdf6-4a13-b939-ac09992e3a36 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:38.098338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:38.161808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52340d-2b6f-4f3d-b29d-7a01f77b8663 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:38.187940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23793796-cdf6-4a13-b939-ac09992e3a36 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:38.188843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:38.199092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80593cab-7d8d-4f28-82ba-11daf1913f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:38.199716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80593cab-7d8d-4f28-82ba-11daf1913f12 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.200003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80593cab-7d8d-4f28-82ba-11daf1913f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.200277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80593cab-7d8d-4f28-82ba-11daf1913f12 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.200978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80593cab-7d8d-4f28-82ba-11daf1913f12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:38.201286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80593cab-7d8d-4f28-82ba-11daf1913f12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:38.202708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80593cab-7d8d-4f28-82ba-11daf1913f12 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:38.214610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 261/501] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:38.215833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d3580-4e80-4326-ab50-13699cad9eb7 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.216125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d3580-4e80-4326-ab50-13699cad9eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.216417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d3580-4e80-4326-ab50-13699cad9eb7 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.217805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 262/502] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:38.218511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d3580-4e80-4326-ab50-13699cad9eb7 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:38.225798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4bbb7-a57e-4b2c-8910-c872ce1d38c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:38.226523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4bbb7-a57e-4b2c-8910-c872ce1d38c5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.227025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4bbb7-a57e-4b2c-8910-c872ce1d38c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.227309 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4bbb7-a57e-4b2c-8910-c872ce1d38c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.302460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4bbb7-a57e-4b2c-8910-c872ce1d38c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:38.303940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4bbb7-a57e-4b2c-8910-c872ce1d38c5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:38.318837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:38.332917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d3580-4e80-4326-ab50-13699cad9eb7 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:38.368470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e7de5-28d2-4a39-b44e-e00d10c62eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:38.369130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e7de5-28d2-4a39-b44e-e00d10c62eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.369460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e7de5-28d2-4a39-b44e-e00d10c62eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.369752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e7de5-28d2-4a39-b44e-e00d10c62eb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.380803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e7de5-28d2-4a39-b44e-e00d10c62eb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-ac3e7de5-28d2-4a39-b44e-e00d10c62eb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:38.386249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:38.387289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:38.388379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:38.392491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 263/503] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:38.393373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e7de5-28d2-4a39-b44e-e00d10c62eb8 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:38.410371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 241/504] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:38.411255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4bbb7-a57e-4b2c-8910-c872ce1d38c5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:38.441975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:38.495627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e7de5-28d2-4a39-b44e-e00d10c62eb8 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:38.513565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4bbb7-a57e-4b2c-8910-c872ce1d38c5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:38.528158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:38.529962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a5692-4e74-4285-bd4c-7ba595317d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:38.530586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5692-4e74-4285-bd4c-7ba595317d5c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.530869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5692-4e74-4285-bd4c-7ba595317d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.531147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5692-4e74-4285-bd4c-7ba595317d5c None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.531855 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a5692-4e74-4285-bd4c-7ba595317d5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:38.532154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a5692-4e74-4285-bd4c-7ba595317d5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:38.533572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a5692-4e74-4285-bd4c-7ba595317d5c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:38.546266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 264/505] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:38.547540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae860047-308e-40f6-835c-d457445e8363 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.547828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae860047-308e-40f6-835c-d457445e8363 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.548101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae860047-308e-40f6-835c-d457445e8363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.549792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eae3c8-41fe-41e9-af0f-7406278180d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:38.554439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eae3c8-41fe-41e9-af0f-7406278180d8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.554439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eae3c8-41fe-41e9-af0f-7406278180d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.554439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eae3c8-41fe-41e9-af0f-7406278180d8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.559181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eae3c8-41fe-41e9-af0f-7406278180d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:38.560678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eae3c8-41fe-41e9-af0f-7406278180d8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:38.567241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-ae860047-308e-40f6-835c-d457445e8363 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:38.568453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ae860047-308e-40f6-835c-d457445e8363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a7eae3c8-41fe-41e9-af0f-7406278180d8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:38.572638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:38.573573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:38.574495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 242/506] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:38.575661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eae3c8-41fe-41e9-af0f-7406278180d8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:38.640116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 265/507] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:38.641274 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae860047-308e-40f6-835c-d457445e8363 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:38.676573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:38.694004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eae3c8-41fe-41e9-af0f-7406278180d8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:38.728275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8b47b-b192-4757-af69-d6df19b92ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:38.728896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8b47b-b192-4757-af69-d6df19b92ecc None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.729185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8b47b-b192-4757-af69-d6df19b92ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.729508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8b47b-b192-4757-af69-d6df19b92ecc None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.730230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8b47b-b192-4757-af69-d6df19b92ecc None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:38.730538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8b47b-b192-4757-af69-d6df19b92ecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:38.731940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8b47b-b192-4757-af69-d6df19b92ecc None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:38.740803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 243/508] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:38.741645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8b47b-b192-4757-af69-d6df19b92ecc None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:38.743974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae860047-308e-40f6-835c-d457445e8363 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:38.778960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:38.780763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2d762-2437-4ae4-a0c4-a3c1fe018619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:38.781450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2d762-2437-4ae4-a0c4-a3c1fe018619 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.781739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2d762-2437-4ae4-a0c4-a3c1fe018619 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.782020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2d762-2437-4ae4-a0c4-a3c1fe018619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.792653 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2d762-2437-4ae4-a0c4-a3c1fe018619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:38.794151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2d762-2437-4ae4-a0c4-a3c1fe018619 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-85d2d762-2437-4ae4-a0c4-a3c1fe018619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:38.810589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:38.811462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:38.818027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:38.818027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:38.818027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:38.818027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:38.818027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 266/509] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:38.818027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2d762-2437-4ae4-a0c4-a3c1fe018619 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:38.848395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8b47b-b192-4757-af69-d6df19b92ecc None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:38.883324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccd43a-62c7-4af1-a74f-f478aea2ff15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:38.883970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd43a-62c7-4af1-a74f-f478aea2ff15 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.884259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd43a-62c7-4af1-a74f-f478aea2ff15 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.884541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd43a-62c7-4af1-a74f-f478aea2ff15 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.893440 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccd43a-62c7-4af1-a74f-f478aea2ff15 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:38.895726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccd43a-62c7-4af1-a74f-f478aea2ff15 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:38.904799 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4eccd43a-62c7-4af1-a74f-f478aea2ff15 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:38.907410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:38.908369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 244/510] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:38.910080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccd43a-62c7-4af1-a74f-f478aea2ff15 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:38.922239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2d762-2437-4ae4-a0c4-a3c1fe018619 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:38.960507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f18f52-496a-4458-98dc-b5d8f29aa1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:38.961143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f18f52-496a-4458-98dc-b5d8f29aa1fd None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:38.961493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f18f52-496a-4458-98dc-b5d8f29aa1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:38.961781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f18f52-496a-4458-98dc-b5d8f29aa1fd None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:38.962592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f18f52-496a-4458-98dc-b5d8f29aa1fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:38.962912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f18f52-496a-4458-98dc-b5d8f29aa1fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:38.964385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f18f52-496a-4458-98dc-b5d8f29aa1fd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:38.982292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 267/511] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:38.983215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f18f52-496a-4458-98dc-b5d8f29aa1fd None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:39.008248 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:39.014893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccd43a-62c7-4af1-a74f-f478aea2ff15 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:39.049652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b95f2d-92ba-431b-9663-2b3881d0dce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:39.050550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95f2d-92ba-431b-9663-2b3881d0dce5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:39.050881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95f2d-92ba-431b-9663-2b3881d0dce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:39.051161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95f2d-92ba-431b-9663-2b3881d0dce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:39.062469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b95f2d-92ba-431b-9663-2b3881d0dce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:39.063972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b95f2d-92ba-431b-9663-2b3881d0dce5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a3b95f2d-92ba-431b-9663-2b3881d0dce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:39.079460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:39.080490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:39.082367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:39.082367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:39.082367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:39.082367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:39.086521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 245/512] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:39.087118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b95f2d-92ba-431b-9663-2b3881d0dce5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:39.089139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f18f52-496a-4458-98dc-b5d8f29aa1fd None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:39.123694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3d9a1-f47e-48d6-ab19-b51ce2cb3dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:39.124333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3d9a1-f47e-48d6-ab19-b51ce2cb3dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:39.124616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3d9a1-f47e-48d6-ab19-b51ce2cb3dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:39.124893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3d9a1-f47e-48d6-ab19-b51ce2cb3dc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:39.136477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3d9a1-f47e-48d6-ab19-b51ce2cb3dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:39.138218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3d9a1-f47e-48d6-ab19-b51ce2cb3dc8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:39.145173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-5df3d9a1-f47e-48d6-ab19-b51ce2cb3dc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:39.153332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:39.160496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:39.161570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:39.161570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:39.161570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:39.161570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:39.161570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 268/513] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:39.161570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3d9a1-f47e-48d6-ab19-b51ce2cb3dc8 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:39.190349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b95f2d-92ba-431b-9663-2b3881d0dce5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:39.225621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd63d55-c363-428b-8c66-47256f6a0387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:39.226234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd63d55-c363-428b-8c66-47256f6a0387 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:39.226513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd63d55-c363-428b-8c66-47256f6a0387 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:39.226786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd63d55-c363-428b-8c66-47256f6a0387 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:39.227477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd63d55-c363-428b-8c66-47256f6a0387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:39.227777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd63d55-c363-428b-8c66-47256f6a0387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:39.229146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd63d55-c363-428b-8c66-47256f6a0387 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:39.241886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 246/514] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:39.242595 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd63d55-c363-428b-8c66-47256f6a0387 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:39.252243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:39.262031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3d9a1-f47e-48d6-ab19-b51ce2cb3dc8 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:39.295645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc336373-fb15-4706-b1b2-f8314aabf9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:39.296267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc336373-fb15-4706-b1b2-f8314aabf9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:39.296553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc336373-fb15-4706-b1b2-f8314aabf9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:39.296835 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc336373-fb15-4706-b1b2-f8314aabf9a1 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:39.297667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc336373-fb15-4706-b1b2-f8314aabf9a1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:39.297977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc336373-fb15-4706-b1b2-f8314aabf9a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:39.299439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc336373-fb15-4706-b1b2-f8314aabf9a1 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:39.317147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 269/515] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 28 06:09:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:39.317927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc336373-fb15-4706-b1b2-f8314aabf9a1 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:39.345536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd63d55-c363-428b-8c66-47256f6a0387 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:39.378879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:39.384823 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fa063-2ae1-4275-8ec9-63fca37d18e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:39.385481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fa063-2ae1-4275-8ec9-63fca37d18e8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:39.385767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fa063-2ae1-4275-8ec9-63fca37d18e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:39.392419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fa063-2ae1-4275-8ec9-63fca37d18e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:39.424088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc336373-fb15-4706-b1b2-f8314aabf9a1 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:39.455924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fa063-2ae1-4275-8ec9-63fca37d18e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:39.457577 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fa063-2ae1-4275-8ec9-63fca37d18e8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:39.458404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf535fd-80fc-4a16-95f7-d380d557654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:39.459029 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf535fd-80fc-4a16-95f7-d380d557654f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:39.459324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf535fd-80fc-4a16-95f7-d380d557654f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:39.459602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf535fd-80fc-4a16-95f7-d380d557654f None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:39.460321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf535fd-80fc-4a16-95f7-d380d557654f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:39.460629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf535fd-80fc-4a16-95f7-d380d557654f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:39.462034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf535fd-80fc-4a16-95f7-d380d557654f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:39.465637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:39.473254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 270/516] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:39.474088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf535fd-80fc-4a16-95f7-d380d557654f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:39.558838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 247/517] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:39.560132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fa063-2ae1-4275-8ec9-63fca37d18e8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:39.576273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf535fd-80fc-4a16-95f7-d380d557654f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:39.592780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:39.609879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2c05c-eabb-4703-853d-97a319db2221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:39.610504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2c05c-eabb-4703-853d-97a319db2221 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:39.610790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2c05c-eabb-4703-853d-97a319db2221 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:39.611127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2c05c-eabb-4703-853d-97a319db2221 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:39.611773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2c05c-eabb-4703-853d-97a319db2221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:39.612079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2c05c-eabb-4703-853d-97a319db2221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:39.614033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2c05c-eabb-4703-853d-97a319db2221 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:39.625447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 271/518] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:39.626259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2c05c-eabb-4703-853d-97a319db2221 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:39.663855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fa063-2ae1-4275-8ec9-63fca37d18e8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:39.671073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:39.698492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808817a4-ea55-4047-837e-7cb33a5a34b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:39.699131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808817a4-ea55-4047-837e-7cb33a5a34b4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:39.699413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808817a4-ea55-4047-837e-7cb33a5a34b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:39.699689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808817a4-ea55-4047-837e-7cb33a5a34b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:39.707425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808817a4-ea55-4047-837e-7cb33a5a34b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:39.708914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808817a4-ea55-4047-837e-7cb33a5a34b4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-808817a4-ea55-4047-837e-7cb33a5a34b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:39.720620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:39.721770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:39.723150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:39.723150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:39.723150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:39.723150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:39.723150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:39.723150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:39.723150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:39.723150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:39.723150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 248/519] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:39.723150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808817a4-ea55-4047-837e-7cb33a5a34b4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:39.734131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2c05c-eabb-4703-853d-97a319db2221 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:39.769513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8b682-c535-4af5-897a-51e5959014d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:39.770144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8b682-c535-4af5-897a-51e5959014d7 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:39.770435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8b682-c535-4af5-897a-51e5959014d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:39.770724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8b682-c535-4af5-897a-51e5959014d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:39.816758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:39.826093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808817a4-ea55-4047-837e-7cb33a5a34b4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:39.840974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8b682-c535-4af5-897a-51e5959014d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:39.842435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8b682-c535-4af5-897a-51e5959014d7 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:39.863325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e70bfa-f120-4ed9-a087-32aa53ebb3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:39.863943 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e70bfa-f120-4ed9-a087-32aa53ebb3aa None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:39.864282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e70bfa-f120-4ed9-a087-32aa53ebb3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:39.864498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e70bfa-f120-4ed9-a087-32aa53ebb3aa None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:39.865177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e70bfa-f120-4ed9-a087-32aa53ebb3aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:39.866658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e70bfa-f120-4ed9-a087-32aa53ebb3aa None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:39.943071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 272/520] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:39.943903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8b682-c535-4af5-897a-51e5959014d7 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:39.973932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 249/521] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:09:39 2022] POST /identity/v3/users => generated 247 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:09:39.974998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e70bfa-f120-4ed9-a087-32aa53ebb3aa None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:40.006299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:40.028778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:40.055142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8b682-c535-4af5-897a-51e5959014d7 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:40.093105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1600d16-dc98-4737-922a-f996c4e28120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:40.093718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1600d16-dc98-4737-922a-f996c4e28120 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:40.093998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1600d16-dc98-4737-922a-f996c4e28120 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:40.094276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1600d16-dc98-4737-922a-f996c4e28120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:40.098485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e70bfa-f120-4ed9-a087-32aa53ebb3aa None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:40.106060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1600d16-dc98-4737-922a-f996c4e28120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:40.107544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1600d16-dc98-4737-922a-f996c4e28120 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-d1600d16-dc98-4737-922a-f996c4e28120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:40.125322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:40.126408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:40.127357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:40.127357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:40.127357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:40.127357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:40.127357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 273/522] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:40.127357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1600d16-dc98-4737-922a-f996c4e28120 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:40.146975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc6a4-d658-4b13-b214-2e43f9334519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:40.147882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc6a4-d658-4b13-b214-2e43f9334519 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:40.148177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc6a4-d658-4b13-b214-2e43f9334519 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:40.148460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc6a4-d658-4b13-b214-2e43f9334519 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:40.156848 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bc6a4-d658-4b13-b214-2e43f9334519 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:40.158393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bc6a4-d658-4b13-b214-2e43f9334519 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c70bc6a4-d658-4b13-b214-2e43f9334519 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:40.171621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:40.172672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 250/523] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:40.173923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc6a4-d658-4b13-b214-2e43f9334519 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:40.249967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1600d16-dc98-4737-922a-f996c4e28120 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:40.273952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:40.280380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:40.290651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a0d6b-e8d8-4cca-a6ce-895404ec79fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:40.291489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a0d6b-e8d8-4cca-a6ce-895404ec79fa None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:40.291775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a0d6b-e8d8-4cca-a6ce-895404ec79fa None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:40.292057 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a0d6b-e8d8-4cca-a6ce-895404ec79fa None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:40.293134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a0d6b-e8d8-4cca-a6ce-895404ec79fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:40.293520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a0d6b-e8d8-4cca-a6ce-895404ec79fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:40.293874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc6a4-d658-4b13-b214-2e43f9334519 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:40.294941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a0d6b-e8d8-4cca-a6ce-895404ec79fa None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:40.318474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 274/524] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:40.318474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a0d6b-e8d8-4cca-a6ce-895404ec79fa None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:40.346239 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02545e-0dab-4445-bbb9-037e9e5749af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:40.347676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02545e-0dab-4445-bbb9-037e9e5749af None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:40.348115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02545e-0dab-4445-bbb9-037e9e5749af None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:40.348491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02545e-0dab-4445-bbb9-037e9e5749af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:40.360079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02545e-0dab-4445-bbb9-037e9e5749af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:40.361780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02545e-0dab-4445-bbb9-037e9e5749af None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-5f02545e-0dab-4445-bbb9-037e9e5749af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:40.377084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:40.378064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:40.379023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:40.379023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:40.379023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:40.379023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:40.379023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 251/525] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:40.380058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02545e-0dab-4445-bbb9-037e9e5749af None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:40.435323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a0d6b-e8d8-4cca-a6ce-895404ec79fa None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:40.470605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4730053-2e45-408c-9ae0-ebc4945b7dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:40.471231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4730053-2e45-408c-9ae0-ebc4945b7dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:40.471522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4730053-2e45-408c-9ae0-ebc4945b7dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:40.472334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4730053-2e45-408c-9ae0-ebc4945b7dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:40.473040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4730053-2e45-408c-9ae0-ebc4945b7dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:40.473380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4730053-2e45-408c-9ae0-ebc4945b7dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:40.474895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4730053-2e45-408c-9ae0-ebc4945b7dd4 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:40.489354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 275/526] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:40.489354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4730053-2e45-408c-9ae0-ebc4945b7dd4 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:40.502213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02545e-0dab-4445-bbb9-037e9e5749af None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:40.538653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669e17b-a084-49db-bcdf-1b81f53cbb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:40.539330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669e17b-a084-49db-bcdf-1b81f53cbb21 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:40.539618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669e17b-a084-49db-bcdf-1b81f53cbb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:40.539892 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669e17b-a084-49db-bcdf-1b81f53cbb21 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:40.540739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669e17b-a084-49db-bcdf-1b81f53cbb21 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:40.541051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669e17b-a084-49db-bcdf-1b81f53cbb21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:40.542508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669e17b-a084-49db-bcdf-1b81f53cbb21 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:40.544182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:40.551088 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:40.587149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 252/527] 10.222.0.69 () {56 vars in 1139 bytes} [Thu Jul 28 06:09:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:40.587149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669e17b-a084-49db-bcdf-1b81f53cbb21 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:40.603586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4730053-2e45-408c-9ae0-ebc4945b7dd4 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:40.651123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64439f17-e3e6-4a6e-9348-18ecb7fa4b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:40.651831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64439f17-e3e6-4a6e-9348-18ecb7fa4b57 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:40.652043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64439f17-e3e6-4a6e-9348-18ecb7fa4b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:40.652327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64439f17-e3e6-4a6e-9348-18ecb7fa4b57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:40.653133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64439f17-e3e6-4a6e-9348-18ecb7fa4b57 None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=5706aa2d6bf54f05badf9588f51f4677) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:40.653476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64439f17-e3e6-4a6e-9348-18ecb7fa4b57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:40.654930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64439f17-e3e6-4a6e-9348-18ecb7fa4b57 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:40.669356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 276/528] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:40 2022] GET /identity/v3/role_assignments?user.id=5706aa2d6bf54f05badf9588f51f4677&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:40.669356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64439f17-e3e6-4a6e-9348-18ecb7fa4b57 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:40.712610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669e17b-a084-49db-bcdf-1b81f53cbb21 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:40.755926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55510e7e-19e2-43de-8598-5750508701cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:40.756722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55510e7e-19e2-43de-8598-5750508701cc None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:40.757083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55510e7e-19e2-43de-8598-5750508701cc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:40.757508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55510e7e-19e2-43de-8598-5750508701cc None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:40.758491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55510e7e-19e2-43de-8598-5750508701cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:40.758872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55510e7e-19e2-43de-8598-5750508701cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:40.761270 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55510e7e-19e2-43de-8598-5750508701cc None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:40.778098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 253/529] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:40.779033 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55510e7e-19e2-43de-8598-5750508701cc None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:40.808325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64439f17-e3e6-4a6e-9348-18ecb7fa4b57 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:40.826345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:40.852484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:40.858747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b290-8e19-4a54-a132-d2ab56ccf0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:40.860051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b290-8e19-4a54-a132-d2ab56ccf0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:40.860669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b290-8e19-4a54-a132-d2ab56ccf0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:40.861188 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b290-8e19-4a54-a132-d2ab56ccf0e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:40.872425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3b290-8e19-4a54-a132-d2ab56ccf0e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:40.878534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3b290-8e19-4a54-a132-d2ab56ccf0e3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-82d3b290-8e19-4a54-a132-d2ab56ccf0e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:40.889564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:40.890618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:40.891712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:40.891712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:40.891712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:40.891712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:40.891712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:40.891712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:40.891712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:40.891712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:40.893249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 277/530] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:40.894650 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b290-8e19-4a54-a132-d2ab56ccf0e3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:40.908119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55510e7e-19e2-43de-8598-5750508701cc None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:40.959455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565ae5e-d00d-49fb-a754-4feea89bd001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:40.960827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565ae5e-d00d-49fb-a754-4feea89bd001 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:40.961440 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565ae5e-d00d-49fb-a754-4feea89bd001 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:40.961870 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565ae5e-d00d-49fb-a754-4feea89bd001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:41.006643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b290-8e19-4a54-a132-d2ab56ccf0e3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:41.051104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b810-3432-4255-a462-90eeb6f91c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:41.052281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b810-3432-4255-a462-90eeb6f91c11 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:41.052722 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b810-3432-4255-a462-90eeb6f91c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:41.053154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b810-3432-4255-a462-90eeb6f91c11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:41.057910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565ae5e-d00d-49fb-a754-4feea89bd001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:41.059536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565ae5e-d00d-49fb-a754-4feea89bd001 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:41.063167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b810-3432-4255-a462-90eeb6f91c11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:41.064796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b810-3432-4255-a462-90eeb6f91c11 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-6ad1b810-3432-4255-a462-90eeb6f91c11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:41.080778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:41.082475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:41.083531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:41.083531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:41.083531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:41.083531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:41.083531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:41.083531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:41.083531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:41.083531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:41.086348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 278/531] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:41.087433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b810-3432-4255-a462-90eeb6f91c11 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:41.124163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:41.133904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:41.182041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 254/532] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:41.183149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565ae5e-d00d-49fb-a754-4feea89bd001 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:41.234056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b810-3432-4255-a462-90eeb6f91c11 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:41.286874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e8dfe-2b2f-4b9c-895e-af9500f9d5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:41.288177 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e8dfe-2b2f-4b9c-895e-af9500f9d5ac None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:41.288860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e8dfe-2b2f-4b9c-895e-af9500f9d5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:41.289294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e8dfe-2b2f-4b9c-895e-af9500f9d5ac None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:41.290493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e8dfe-2b2f-4b9c-895e-af9500f9d5ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:41.291398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e8dfe-2b2f-4b9c-895e-af9500f9d5ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:41.293552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e8dfe-2b2f-4b9c-895e-af9500f9d5ac None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:41.297232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565ae5e-d00d-49fb-a754-4feea89bd001 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:41.310416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 279/533] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:41.311565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e8dfe-2b2f-4b9c-895e-af9500f9d5ac None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:41.342309 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7cd99-c727-496c-9706-6396cf2f3a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:41.343087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7cd99-c727-496c-9706-6396cf2f3a85 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:41.343462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7cd99-c727-496c-9706-6396cf2f3a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:41.343836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7cd99-c727-496c-9706-6396cf2f3a85 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:41.344735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7cd99-c727-496c-9706-6396cf2f3a85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:41.345166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7cd99-c727-496c-9706-6396cf2f3a85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:41.348568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7cd99-c727-496c-9706-6396cf2f3a85 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:41.379502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 255/534] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:41.380073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7cd99-c727-496c-9706-6396cf2f3a85 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:41.397541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:41.443925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:41.455633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e8dfe-2b2f-4b9c-895e-af9500f9d5ac None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:41.496387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54619f95-3d31-44ae-a83c-a63eb9410e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:41.497101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54619f95-3d31-44ae-a83c-a63eb9410e07 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:41.497476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54619f95-3d31-44ae-a83c-a63eb9410e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:41.497762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54619f95-3d31-44ae-a83c-a63eb9410e07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:41.511913 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54619f95-3d31-44ae-a83c-a63eb9410e07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:41.514386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54619f95-3d31-44ae-a83c-a63eb9410e07 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:41.516417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7cd99-c727-496c-9706-6396cf2f3a85 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-54619f95-3d31-44ae-a83c-a63eb9410e07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:41.537549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:41.538483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:41.540026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:41.540026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:41.540026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:41.540026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:41.540026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 280/535] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:41.540026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54619f95-3d31-44ae-a83c-a63eb9410e07 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:41.566632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3f1ee-175e-4fe1-8fb6-fc5425a9b2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:41.567641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3f1ee-175e-4fe1-8fb6-fc5425a9b2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:41.568130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3f1ee-175e-4fe1-8fb6-fc5425a9b2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:41.568610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3f1ee-175e-4fe1-8fb6-fc5425a9b2d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:41.570050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3f1ee-175e-4fe1-8fb6-fc5425a9b2d8 None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=cc4fe04b11ec412e8f4eb67c6e54b8f7) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:41.570373 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3f1ee-175e-4fe1-8fb6-fc5425a9b2d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:41.571824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3f1ee-175e-4fe1-8fb6-fc5425a9b2d8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:41.582818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 256/536] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:41 2022] GET /identity/v3/role_assignments?user.id=cc4fe04b11ec412e8f4eb67c6e54b8f7&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:41.583773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3f1ee-175e-4fe1-8fb6-fc5425a9b2d8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:41.664090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54619f95-3d31-44ae-a83c-a63eb9410e07 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:41.709746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d332a73-8ee1-4fb1-832a-afa96d28ca74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:41.711727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d332a73-8ee1-4fb1-832a-afa96d28ca74 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:41.712996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d332a73-8ee1-4fb1-832a-afa96d28ca74 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:41.713697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d332a73-8ee1-4fb1-832a-afa96d28ca74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:41.715928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d332a73-8ee1-4fb1-832a-afa96d28ca74 None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=cbbbf34c238d44db8588d56f35827390) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:41.716741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d332a73-8ee1-4fb1-832a-afa96d28ca74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:41.719151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d332a73-8ee1-4fb1-832a-afa96d28ca74 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:41.732250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 281/537] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:41 2022] GET /identity/v3/role_assignments?user.id=cbbbf34c238d44db8588d56f35827390&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:41.751050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:41.786200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3f1ee-175e-4fe1-8fb6-fc5425a9b2d8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:41.832936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a454ee-c545-4e69-8de0-115e7e537030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:41.834032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a454ee-c545-4e69-8de0-115e7e537030 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:41.834592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a454ee-c545-4e69-8de0-115e7e537030 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:41.835083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a454ee-c545-4e69-8de0-115e7e537030 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:41.836085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a454ee-c545-4e69-8de0-115e7e537030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:41.836646 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a454ee-c545-4e69-8de0-115e7e537030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:41.838376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a454ee-c545-4e69-8de0-115e7e537030 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:41.854499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 257/538] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:41.859930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d332a73-8ee1-4fb1-832a-afa96d28ca74 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:41.891986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:41.985725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d332a73-8ee1-4fb1-832a-afa96d28ca74 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:42.020844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905acd9-ec58-4f7a-b55b-0cdd9cdcc899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:42.021532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905acd9-ec58-4f7a-b55b-0cdd9cdcc899 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:42.021831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905acd9-ec58-4f7a-b55b-0cdd9cdcc899 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:42.022117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905acd9-ec58-4f7a-b55b-0cdd9cdcc899 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:42.029173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905acd9-ec58-4f7a-b55b-0cdd9cdcc899 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:42.030690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905acd9-ec58-4f7a-b55b-0cdd9cdcc899 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-1905acd9-ec58-4f7a-b55b-0cdd9cdcc899 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:42.051679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:42.052564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:42.053437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:42.053437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:42.053437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:42.053437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:42.053437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:42.053437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:42.053437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:42.053437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:42.053437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 282/539] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:42.060026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a454ee-c545-4e69-8de0-115e7e537030 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:42.119503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d4da7-3e57-4a18-8889-c63b9542ee39 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:42.119813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d4da7-3e57-4a18-8889-c63b9542ee39 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:42.120070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d4da7-3e57-4a18-8889-c63b9542ee39 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:42.121745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 283/540] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:42.130991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1acf4-bd46-4f13-b889-0b7cf8bb838d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:42.131475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1acf4-bd46-4f13-b889-0b7cf8bb838d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:42.131885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1acf4-bd46-4f13-b889-0b7cf8bb838d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:42.172750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-42d1acf4-bd46-4f13-b889-0b7cf8bb838d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:42.174255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-42d1acf4-bd46-4f13-b889-0b7cf8bb838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:42.259348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a454ee-c545-4e69-8de0-115e7e537030 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:42.266097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 284/541] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:42.269763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:42.296184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1acf4-bd46-4f13-b889-0b7cf8bb838d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:42.341370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495c807-15f3-4474-92b2-2a47924d4696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:42.342032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c807-15f3-4474-92b2-2a47924d4696 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:42.342323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c807-15f3-4474-92b2-2a47924d4696 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:42.342607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c807-15f3-4474-92b2-2a47924d4696 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:42.343385 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495c807-15f3-4474-92b2-2a47924d4696 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:42.343704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495c807-15f3-4474-92b2-2a47924d4696 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:42.345169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495c807-15f3-4474-92b2-2a47924d4696 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:42.356070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:42.379777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 258/542] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:42.389934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495c807-15f3-4474-92b2-2a47924d4696 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:42.420316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1acf4-bd46-4f13-b889-0b7cf8bb838d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:42.463071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46dddc-5ffd-4ebd-a265-363ddd438278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:42.464525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46dddc-5ffd-4ebd-a265-363ddd438278 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:42.465015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46dddc-5ffd-4ebd-a265-363ddd438278 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:42.465557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46dddc-5ffd-4ebd-a265-363ddd438278 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:42.479615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46dddc-5ffd-4ebd-a265-363ddd438278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-fb46dddc-5ffd-4ebd-a265-363ddd438278 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:42.485530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:42.486421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:42.487736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:42.490666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 285/543] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:42.496532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46dddc-5ffd-4ebd-a265-363ddd438278 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:42.632438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:42.710505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46dddc-5ffd-4ebd-a265-363ddd438278 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:42.752155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495c807-15f3-4474-92b2-2a47924d4696 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:42.796378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f526e-accd-428b-a79c-cb277e79bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:42.797016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f526e-accd-428b-a79c-cb277e79bcbd None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:42.797299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f526e-accd-428b-a79c-cb277e79bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:42.797595 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f526e-accd-428b-a79c-cb277e79bcbd None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:42.798326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f526e-accd-428b-a79c-cb277e79bcbd None admin] RBAC: Adding query filter params (name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:42.798631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f526e-accd-428b-a79c-cb277e79bcbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:42.800127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f526e-accd-428b-a79c-cb277e79bcbd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:42.823216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 286/544] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:42.826004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f526e-accd-428b-a79c-cb277e79bcbd None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:42.838797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f59625-4550-4259-8073-c40dfbe8cef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:42.844294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f59625-4550-4259-8073-c40dfbe8cef8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:42.844294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f59625-4550-4259-8073-c40dfbe8cef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:42.844294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f59625-4550-4259-8073-c40dfbe8cef8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:42.844294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f59625-4550-4259-8073-c40dfbe8cef8 None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=05574eace38f4b6d945c7b1927fdadd8) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:42.844294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f59625-4550-4259-8073-c40dfbe8cef8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:42.844294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f59625-4550-4259-8073-c40dfbe8cef8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:42.862084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 259/545] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:42 2022] GET /identity/v3/role_assignments?user.id=05574eace38f4b6d945c7b1927fdadd8&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 259 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:42.883005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb37cc0-116a-46f1-8b95-9d3bf2a420a7 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:42.883286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb37cc0-116a-46f1-8b95-9d3bf2a420a7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:42.883614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb37cc0-116a-46f1-8b95-9d3bf2a420a7 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:42.884975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 260/546] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:42.902895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e7868-49da-4aff-9e99-9decd7a82fe4 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:42.903216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e7868-49da-4aff-9e99-9decd7a82fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:42.903464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e7868-49da-4aff-9e99-9decd7a82fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:42.956380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bb9e7868-49da-4aff-9e99-9decd7a82fe4 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:42.958199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bb9e7868-49da-4aff-9e99-9decd7a82fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:43.034170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f526e-accd-428b-a79c-cb277e79bcbd None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:43.085336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:43.122306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:43.125874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8f719-bf18-4c99-8270-5e94cfc821c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:43.126553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8f719-bf18-4c99-8270-5e94cfc821c8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:43.126840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8f719-bf18-4c99-8270-5e94cfc821c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:43.128399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8f719-bf18-4c99-8270-5e94cfc821c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:43.152318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8f719-bf18-4c99-8270-5e94cfc821c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:43.154430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8f719-bf18-4c99-8270-5e94cfc821c8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-eea8f719-bf18-4c99-8270-5e94cfc821c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:43.176947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:43.177961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:43.178786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:43.178786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:43.178786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:43.178786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:43.190525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 287/547] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:43.198288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 261/548] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:43.200811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8f719-bf18-4c99-8270-5e94cfc821c8 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:43.221209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e7868-49da-4aff-9e99-9decd7a82fe4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:43.406043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8f719-bf18-4c99-8270-5e94cfc821c8 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:43.435721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e7868-49da-4aff-9e99-9decd7a82fe4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:43.470438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc0553-3a3a-4958-ac14-39f5712876a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:43.473091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc0553-3a3a-4958-ac14-39f5712876a9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:43.476836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc0553-3a3a-4958-ac14-39f5712876a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:43.476836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc0553-3a3a-4958-ac14-39f5712876a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:43.476836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc0553-3a3a-4958-ac14-39f5712876a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:43.478302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc0553-3a3a-4958-ac14-39f5712876a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:43.479933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc0553-3a3a-4958-ac14-39f5712876a9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:43.508035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 288/549] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:43.509049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148b0a3-3cad-41d2-abe3-2396333f695a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:43.510110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148b0a3-3cad-41d2-abe3-2396333f695a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:43.510634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148b0a3-3cad-41d2-abe3-2396333f695a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:43.511140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148b0a3-3cad-41d2-abe3-2396333f695a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:43.514521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc0553-3a3a-4958-ac14-39f5712876a9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:43.537535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148b0a3-3cad-41d2-abe3-2396333f695a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-f148b0a3-3cad-41d2-abe3-2396333f695a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:43.546893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:43.547840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:43.548641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:43.555932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 262/550] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:43.559217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148b0a3-3cad-41d2-abe3-2396333f695a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:43.585707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:43.672328 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:43.732836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc0553-3a3a-4958-ac14-39f5712876a9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:43.776651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b16aeb-bd9c-4406-860c-00a96cb56e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:43.777938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16aeb-bd9c-4406-860c-00a96cb56e76 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:43.778261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16aeb-bd9c-4406-860c-00a96cb56e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:43.780408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16aeb-bd9c-4406-860c-00a96cb56e76 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:43.793256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b16aeb-bd9c-4406-860c-00a96cb56e76 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:43.796888 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148b0a3-3cad-41d2-abe3-2396333f695a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:43.798967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b16aeb-bd9c-4406-860c-00a96cb56e76 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-40b16aeb-bd9c-4406-860c-00a96cb56e76 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:43.821727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:43.822696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:43.823602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:43.823602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:43.823602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:43.823602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:43.823602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:43.823602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:43.823602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:43.823602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:43.823602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:09:43.823602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:43.829345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 289/551] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:09:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:43.833443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b16aeb-bd9c-4406-860c-00a96cb56e76 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:43.872157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf7e3-8c5b-4c2e-9b95-e128c9cc74f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:43.872785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf7e3-8c5b-4c2e-9b95-e128c9cc74f6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:43.873929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf7e3-8c5b-4c2e-9b95-e128c9cc74f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:43.874222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf7e3-8c5b-4c2e-9b95-e128c9cc74f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:43.874960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaf7e3-8c5b-4c2e-9b95-e128c9cc74f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:43.875272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaf7e3-8c5b-4c2e-9b95-e128c9cc74f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:43.876670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaf7e3-8c5b-4c2e-9b95-e128c9cc74f6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:43.894716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 263/552] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:43.899074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf7e3-8c5b-4c2e-9b95-e128c9cc74f6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:44.005868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:44.022727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b16aeb-bd9c-4406-860c-00a96cb56e76 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:44.099752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf7e3-8c5b-4c2e-9b95-e128c9cc74f6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:44.100873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15855dbe-ad1d-4875-a3b2-3f1f912fda31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:44.101540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15855dbe-ad1d-4875-a3b2-3f1f912fda31 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.101831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15855dbe-ad1d-4875-a3b2-3f1f912fda31 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.102115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15855dbe-ad1d-4875-a3b2-3f1f912fda31 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.102934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15855dbe-ad1d-4875-a3b2-3f1f912fda31 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:44.103249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15855dbe-ad1d-4875-a3b2-3f1f912fda31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:44.104715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15855dbe-ad1d-4875-a3b2-3f1f912fda31 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:44.115297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:44.129047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 290/553] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 28 06:09:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:44.130528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cb270-6145-43f5-837c-371e4a2a4e24 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.130824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cb270-6145-43f5-837c-371e4a2a4e24 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.131106 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cb270-6145-43f5-837c-371e4a2a4e24 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.132893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 291/554] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:44.133709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cb270-6145-43f5-837c-371e4a2a4e24 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:44.151080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdaff1-269e-4779-b375-786a9ae8ec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:44.151852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdaff1-269e-4779-b375-786a9ae8ec3e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.152140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdaff1-269e-4779-b375-786a9ae8ec3e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.152418 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdaff1-269e-4779-b375-786a9ae8ec3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.165711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdaff1-269e-4779-b375-786a9ae8ec3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:44.167305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdaff1-269e-4779-b375-786a9ae8ec3e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b6cdaff1-269e-4779-b375-786a9ae8ec3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:44.185745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:44.186664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:44.187531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:44.187531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:44.187531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:44.187531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:44.188168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 264/555] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:44.189616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32c122-e4eb-41e8-8c2d-85a8601b93a7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.193343 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32c122-e4eb-41e8-8c2d-85a8601b93a7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.193343 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32c122-e4eb-41e8-8c2d-85a8601b93a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.215403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-be32c122-e4eb-41e8-8c2d-85a8601b93a7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:44.216616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-be32c122-e4eb-41e8-8c2d-85a8601b93a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:44.254288 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cb270-6145-43f5-837c-371e4a2a4e24 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:44.300126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 265/556] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:44.301438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32c122-e4eb-41e8-8c2d-85a8601b93a7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:44.306182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a425b3b-b333-46b2-af78-9292e6f6296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:44.306814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a425b3b-b333-46b2-af78-9292e6f6296a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.307096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a425b3b-b333-46b2-af78-9292e6f6296a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.307374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a425b3b-b333-46b2-af78-9292e6f6296a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.331941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:44.396589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a425b3b-b333-46b2-af78-9292e6f6296a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:44.398211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a425b3b-b333-46b2-af78-9292e6f6296a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:44.422384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:44.430345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32c122-e4eb-41e8-8c2d-85a8601b93a7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:44.469904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f7c03-0f33-495d-944f-150ee054a174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:44.471302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f7c03-0f33-495d-944f-150ee054a174 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.472095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f7c03-0f33-495d-944f-150ee054a174 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.472560 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f7c03-0f33-495d-944f-150ee054a174 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.473281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f7c03-0f33-495d-944f-150ee054a174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:44.473624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f7c03-0f33-495d-944f-150ee054a174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:44.475096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f7c03-0f33-495d-944f-150ee054a174 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:44.487217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 266/557] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:44.489130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3cd8-56a7-4da0-8509-1bf2fc57a3db None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.489464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3cd8-56a7-4da0-8509-1bf2fc57a3db None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.489747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3cd8-56a7-4da0-8509-1bf2fc57a3db None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.495594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 267/558] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:44.495594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f3cd8-56a7-4da0-8509-1bf2fc57a3db None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:44.520560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 292/559] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:09:44.521372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a425b3b-b333-46b2-af78-9292e6f6296a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:44.578523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:44.610285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f3cd8-56a7-4da0-8509-1bf2fc57a3db None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:44.634620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a425b3b-b333-46b2-af78-9292e6f6296a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:44.648526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746fd32-335c-4499-ace2-7063d3234c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:44.649174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746fd32-335c-4499-ace2-7063d3234c9d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.649704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746fd32-335c-4499-ace2-7063d3234c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.649864 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746fd32-335c-4499-ace2-7063d3234c9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.662092 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746fd32-335c-4499-ace2-7063d3234c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-1746fd32-335c-4499-ace2-7063d3234c9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:44.668035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:44.668947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:44.670159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 268/560] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:44.671175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ad2d2-45a4-4234-af16-e3be8715976a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.671175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ad2d2-45a4-4234-af16-e3be8715976a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.671175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ad2d2-45a4-4234-af16-e3be8715976a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.680712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28e0af-5bf3-44f5-8b9c-8e467e5639e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:44.681399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28e0af-5bf3-44f5-8b9c-8e467e5639e9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.681694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28e0af-5bf3-44f5-8b9c-8e467e5639e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.681979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28e0af-5bf3-44f5-8b9c-8e467e5639e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.689850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28e0af-5bf3-44f5-8b9c-8e467e5639e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:44.689985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-481ad2d2-45a4-4234-af16-e3be8715976a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:44.691693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-481ad2d2-45a4-4234-af16-e3be8715976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:44.691803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28e0af-5bf3-44f5-8b9c-8e467e5639e9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:44.702003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-1c28e0af-5bf3-44f5-8b9c-8e467e5639e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:44.703268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:44.704378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 293/561] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 28 06:09:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:44.705636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28e0af-5bf3-44f5-8b9c-8e467e5639e9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:44.781333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 269/562] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:44.781333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ad2d2-45a4-4234-af16-e3be8715976a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:44.820520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28e0af-5bf3-44f5-8b9c-8e467e5639e9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:44.833084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:44.857797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f5189-9568-415e-8399-d56a7ad0b145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:44.858455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f5189-9568-415e-8399-d56a7ad0b145 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.858738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f5189-9568-415e-8399-d56a7ad0b145 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.859047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f5189-9568-415e-8399-d56a7ad0b145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.872093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f5189-9568-415e-8399-d56a7ad0b145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:44.873601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f5189-9568-415e-8399-d56a7ad0b145 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-5c7f5189-9568-415e-8399-d56a7ad0b145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:44.889649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:44.890683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:44.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:44.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:44.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:44.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:44.891574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 294/563] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:44.892372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f5189-9568-415e-8399-d56a7ad0b145 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:44.901067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ad2d2-45a4-4234-af16-e3be8715976a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:44.954145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555df9e-dcf2-44b5-b303-422a8d4c21e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:44.955301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555df9e-dcf2-44b5-b303-422a8d4c21e7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:44.955701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555df9e-dcf2-44b5-b303-422a8d4c21e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:44.956161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555df9e-dcf2-44b5-b303-422a8d4c21e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:44.957016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555df9e-dcf2-44b5-b303-422a8d4c21e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:44.957557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555df9e-dcf2-44b5-b303-422a8d4c21e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:44.959268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555df9e-dcf2-44b5-b303-422a8d4c21e7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:44.968917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 270/564] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:44.970335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555df9e-dcf2-44b5-b303-422a8d4c21e7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:44.983729 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:45.026984 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f5189-9568-415e-8399-d56a7ad0b145 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.062374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d479d-9ef7-4f06-a825-f3cf2a06a0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.062995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d479d-9ef7-4f06-a825-f3cf2a06a0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.063290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d479d-9ef7-4f06-a825-f3cf2a06a0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.063573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d479d-9ef7-4f06-a825-f3cf2a06a0e0 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.064385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d479d-9ef7-4f06-a825-f3cf2a06a0e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:45.064699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d479d-9ef7-4f06-a825-f3cf2a06a0e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:45.066228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d479d-9ef7-4f06-a825-f3cf2a06a0e0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:45.083904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555df9e-dcf2-44b5-b303-422a8d4c21e7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.085433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 295/565] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 28 06:09:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:45.086199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d479d-9ef7-4f06-a825-f3cf2a06a0e0 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.108684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:45.120054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57489559-2e65-4fe3-8a72-4620e3cd4294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.120784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489559-2e65-4fe3-8a72-4620e3cd4294 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.121071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489559-2e65-4fe3-8a72-4620e3cd4294 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.121379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489559-2e65-4fe3-8a72-4620e3cd4294 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.132254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57489559-2e65-4fe3-8a72-4620e3cd4294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-57489559-2e65-4fe3-8a72-4620e3cd4294 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:45.137894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:45.138997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 271/566] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:45.140021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57489559-2e65-4fe3-8a72-4620e3cd4294 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.193007 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d479d-9ef7-4f06-a825-f3cf2a06a0e0 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.206507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:45.227480 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e7e6b-fb72-4c4f-a90b-f4ed5bd69889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.228248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e7e6b-fb72-4c4f-a90b-f4ed5bd69889 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.228635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e7e6b-fb72-4c4f-a90b-f4ed5bd69889 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.229032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e7e6b-fb72-4c4f-a90b-f4ed5bd69889 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.229870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e7e6b-fb72-4c4f-a90b-f4ed5bd69889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:45.230278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e7e6b-fb72-4c4f-a90b-f4ed5bd69889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:45.231757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e7e6b-fb72-4c4f-a90b-f4ed5bd69889 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:45.243309 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 296/567] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:45.244114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57489559-2e65-4fe3-8a72-4620e3cd4294 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.244516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdbcb0-db1b-4b84-bfe2-ecfa06c6b979 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.244805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdbcb0-db1b-4b84-bfe2-ecfa06c6b979 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.245086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdbcb0-db1b-4b84-bfe2-ecfa06c6b979 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.250259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 297/568] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:45.251057 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdbcb0-db1b-4b84-bfe2-ecfa06c6b979 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.279946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e84b3d-7d64-4f68-9ab0-9c7eb67af2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.280617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e84b3d-7d64-4f68-9ab0-9c7eb67af2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.280902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e84b3d-7d64-4f68-9ab0-9c7eb67af2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.281178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e84b3d-7d64-4f68-9ab0-9c7eb67af2e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.291909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e84b3d-7d64-4f68-9ab0-9c7eb67af2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:45.293798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e84b3d-7d64-4f68-9ab0-9c7eb67af2e0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d4e84b3d-7d64-4f68-9ab0-9c7eb67af2e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:45.309076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:45.310176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:45.311249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:45.311249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:45.311249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:45.311249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:45.311249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 272/569] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:45.311249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e84b3d-7d64-4f68-9ab0-9c7eb67af2e0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.336639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:45.357425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdbcb0-db1b-4b84-bfe2-ecfa06c6b979 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.393914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d077d-30ac-4bee-a62f-e311cb370e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.394569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d077d-30ac-4bee-a62f-e311cb370e14 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.394854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d077d-30ac-4bee-a62f-e311cb370e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.395132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d077d-30ac-4bee-a62f-e311cb370e14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.405360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d077d-30ac-4bee-a62f-e311cb370e14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:45.406826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d077d-30ac-4bee-a62f-e311cb370e14 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:45.420252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e84b3d-7d64-4f68-9ab0-9c7eb67af2e0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-990d077d-30ac-4bee-a62f-e311cb370e14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:45.422781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:45.423821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:45.425764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:45.425764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:45.425764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:45.425764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:45.425764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 298/570] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:45.425764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d077d-30ac-4bee-a62f-e311cb370e14 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.444694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:45.461963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f1ccb-920f-4c67-8e9f-e816410d8cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.462585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1ccb-920f-4c67-8e9f-e816410d8cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.462876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1ccb-920f-4c67-8e9f-e816410d8cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.463156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1ccb-920f-4c67-8e9f-e816410d8cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.463877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f1ccb-920f-4c67-8e9f-e816410d8cc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:45.464184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f1ccb-920f-4c67-8e9f-e816410d8cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:45.465616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f1ccb-920f-4c67-8e9f-e816410d8cc0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:45.474578 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 273/571] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:45.475812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af4847-b673-4421-a6c2-a544099c41c7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.476497 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af4847-b673-4421-a6c2-a544099c41c7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.477145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af4847-b673-4421-a6c2-a544099c41c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.495724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-88af4847-b673-4421-a6c2-a544099c41c7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:45.496922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-88af4847-b673-4421-a6c2-a544099c41c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:45.544781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d077d-30ac-4bee-a62f-e311cb370e14 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.573292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:45.577749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 274/572] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:45.578226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d421a7c-af31-4b2a-a65f-4f0a35b36d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.578585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af4847-b673-4421-a6c2-a544099c41c7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.579016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d421a7c-af31-4b2a-a65f-4f0a35b36d32 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.579245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d421a7c-af31-4b2a-a65f-4f0a35b36d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.579532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d421a7c-af31-4b2a-a65f-4f0a35b36d32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.600448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d421a7c-af31-4b2a-a65f-4f0a35b36d32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:45.601900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d421a7c-af31-4b2a-a65f-4f0a35b36d32 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-7d421a7c-af31-4b2a-a65f-4f0a35b36d32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:45.614979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:45.615999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:45.616902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:45.616902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:45.616902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:45.616902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:45.616902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:45.616902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:45.616902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:45.616902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:45.621009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 299/573] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:45.621729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d421a7c-af31-4b2a-a65f-4f0a35b36d32 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.677521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af4847-b673-4421-a6c2-a544099c41c7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.688021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:45.712286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936e917-c636-4a77-b5c0-dfdb14a48ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.712859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936e917-c636-4a77-b5c0-dfdb14a48ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.713147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936e917-c636-4a77-b5c0-dfdb14a48ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.713465 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936e917-c636-4a77-b5c0-dfdb14a48ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.714170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936e917-c636-4a77-b5c0-dfdb14a48ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:45.714484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936e917-c636-4a77-b5c0-dfdb14a48ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:45.715872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936e917-c636-4a77-b5c0-dfdb14a48ea7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:45.724194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d421a7c-af31-4b2a-a65f-4f0a35b36d32 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.727855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 275/574] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:45.728660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936e917-c636-4a77-b5c0-dfdb14a48ea7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.757956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bf859-3946-473b-9eaf-37c5882bf1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.758590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bf859-3946-473b-9eaf-37c5882bf1cd None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.758878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bf859-3946-473b-9eaf-37c5882bf1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.759163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bf859-3946-473b-9eaf-37c5882bf1cd None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.759866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bf859-3946-473b-9eaf-37c5882bf1cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:45.760178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bf859-3946-473b-9eaf-37c5882bf1cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:45.761597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bf859-3946-473b-9eaf-37c5882bf1cd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:45.773477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 300/575] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:45.774278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bf859-3946-473b-9eaf-37c5882bf1cd None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.826380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:45.836744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936e917-c636-4a77-b5c0-dfdb14a48ea7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.871012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ac2be-6413-4505-ae69-953be1412e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.871581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ac2be-6413-4505-ae69-953be1412e1a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.871872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ac2be-6413-4505-ae69-953be1412e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.872158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ac2be-6413-4505-ae69-953be1412e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.874865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bf859-3946-473b-9eaf-37c5882bf1cd None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:45.882826 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ac2be-6413-4505-ae69-953be1412e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:45.884278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ac2be-6413-4505-ae69-953be1412e1a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-081ac2be-6413-4505-ae69-953be1412e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:45.898702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:45.899733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:45.900615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:45.900615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:45.900615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:45.900615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:45.904581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 276/576] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:45.905432 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ac2be-6413-4505-ae69-953be1412e1a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.911208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74ef3e-4af5-4643-9be9-2690b3989740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:45.911865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74ef3e-4af5-4643-9be9-2690b3989740 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:45.912155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74ef3e-4af5-4643-9be9-2690b3989740 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:45.912439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74ef3e-4af5-4643-9be9-2690b3989740 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:45.923052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74ef3e-4af5-4643-9be9-2690b3989740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-bb74ef3e-4af5-4643-9be9-2690b3989740 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:45.928698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:45.930163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 301/577] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:45.931745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74ef3e-4af5-4643-9be9-2690b3989740 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:45.936185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:46.013477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ac2be-6413-4505-ae69-953be1412e1a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.034795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74ef3e-4af5-4643-9be9-2690b3989740 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.047499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61034ac6-41f1-4866-b8ec-95a8df1aed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.048113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61034ac6-41f1-4866-b8ec-95a8df1aed6e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.048396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61034ac6-41f1-4866-b8ec-95a8df1aed6e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.048671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61034ac6-41f1-4866-b8ec-95a8df1aed6e None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.048776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:46.049473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61034ac6-41f1-4866-b8ec-95a8df1aed6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:46.049850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61034ac6-41f1-4866-b8ec-95a8df1aed6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.051296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61034ac6-41f1-4866-b8ec-95a8df1aed6e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.063189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 277/578] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:46.063899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61034ac6-41f1-4866-b8ec-95a8df1aed6e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:46.071686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef726b18-163f-4e1a-b5a1-da06eb45f257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.072334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef726b18-163f-4e1a-b5a1-da06eb45f257 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.072627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef726b18-163f-4e1a-b5a1-da06eb45f257 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.073524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef726b18-163f-4e1a-b5a1-da06eb45f257 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.082060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef726b18-163f-4e1a-b5a1-da06eb45f257 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.083773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef726b18-163f-4e1a-b5a1-da06eb45f257 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-ef726b18-163f-4e1a-b5a1-da06eb45f257 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:46.095408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:46.097039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 302/579] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:46.098027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef726b18-163f-4e1a-b5a1-da06eb45f257 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:46.155767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:46.173845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61034ac6-41f1-4866-b8ec-95a8df1aed6e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.200109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef726b18-163f-4e1a-b5a1-da06eb45f257 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.207699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f675e8-5735-4de8-9f34-fbf034642155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.208339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f675e8-5735-4de8-9f34-fbf034642155 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.208624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f675e8-5735-4de8-9f34-fbf034642155 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.208901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f675e8-5735-4de8-9f34-fbf034642155 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.215473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f675e8-5735-4de8-9f34-fbf034642155 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.216923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f675e8-5735-4de8-9f34-fbf034642155 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e2f675e8-5735-4de8-9f34-fbf034642155 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:46.232232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:46.233380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:46.234699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:46.234699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:46.234699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:46.234699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:46.234699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:46.234699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:46.234699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:46.234699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:46.234699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 278/580] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:46.234699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f675e8-5735-4de8-9f34-fbf034642155 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:46.240238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f925dd-acb3-49a2-8d92-3facd3db1bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.240877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f925dd-acb3-49a2-8d92-3facd3db1bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.241157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f925dd-acb3-49a2-8d92-3facd3db1bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.241463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f925dd-acb3-49a2-8d92-3facd3db1bf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.242170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f925dd-acb3-49a2-8d92-3facd3db1bf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:46.242476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f925dd-acb3-49a2-8d92-3facd3db1bf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.243841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f925dd-acb3-49a2-8d92-3facd3db1bf6 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.255895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 303/581] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:46.255895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f925dd-acb3-49a2-8d92-3facd3db1bf6 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:46.294790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:46.336939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f675e8-5735-4de8-9f34-fbf034642155 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.366230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f925dd-acb3-49a2-8d92-3facd3db1bf6 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.372200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e60551-5cd5-4a40-8e32-1d52a9f806c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.372829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e60551-5cd5-4a40-8e32-1d52a9f806c4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.373120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e60551-5cd5-4a40-8e32-1d52a9f806c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.373463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e60551-5cd5-4a40-8e32-1d52a9f806c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.374190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e60551-5cd5-4a40-8e32-1d52a9f806c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:46.374503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e60551-5cd5-4a40-8e32-1d52a9f806c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.375880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e60551-5cd5-4a40-8e32-1d52a9f806c4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.384089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 279/582] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:46.384919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e60551-5cd5-4a40-8e32-1d52a9f806c4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:46.388619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:46.404352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6e59d-93ee-4b2c-9959-c7495fbbe154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.404985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6e59d-93ee-4b2c-9959-c7495fbbe154 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.405272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6e59d-93ee-4b2c-9959-c7495fbbe154 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.405587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6e59d-93ee-4b2c-9959-c7495fbbe154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.406453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6e59d-93ee-4b2c-9959-c7495fbbe154 None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=77571de30e934d6590ff7b58a000f5e2) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:46.406715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6e59d-93ee-4b2c-9959-c7495fbbe154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.408168 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6e59d-93ee-4b2c-9959-c7495fbbe154 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.415067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 304/583] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:46 2022] GET /identity/v3/role_assignments?user.id=77571de30e934d6590ff7b58a000f5e2&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:46.415067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6e59d-93ee-4b2c-9959-c7495fbbe154 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:46.518920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e60551-5cd5-4a40-8e32-1d52a9f806c4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.567433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:46.573505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6e59d-93ee-4b2c-9959-c7495fbbe154 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.577962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e435ff-c57c-4fc6-8fcb-a8db3ec71c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.578589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e435ff-c57c-4fc6-8fcb-a8db3ec71c73 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.578871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e435ff-c57c-4fc6-8fcb-a8db3ec71c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.579149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e435ff-c57c-4fc6-8fcb-a8db3ec71c73 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.579953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e435ff-c57c-4fc6-8fcb-a8db3ec71c73 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:46.580257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e435ff-c57c-4fc6-8fcb-a8db3ec71c73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.582526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e435ff-c57c-4fc6-8fcb-a8db3ec71c73 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.600718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 280/584] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 28 06:09:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:46.601993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e435ff-c57c-4fc6-8fcb-a8db3ec71c73 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:46.617693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763d0b1-4f58-471a-ae0f-3f2c6c254e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.618326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763d0b1-4f58-471a-ae0f-3f2c6c254e83 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.618625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763d0b1-4f58-471a-ae0f-3f2c6c254e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.618909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763d0b1-4f58-471a-ae0f-3f2c6c254e83 None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.619675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763d0b1-4f58-471a-ae0f-3f2c6c254e83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:46.619991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763d0b1-4f58-471a-ae0f-3f2c6c254e83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.621471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763d0b1-4f58-471a-ae0f-3f2c6c254e83 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.634801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 305/585] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:46.636077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763d0b1-4f58-471a-ae0f-3f2c6c254e83 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:46.673624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:46.719758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e435ff-c57c-4fc6-8fcb-a8db3ec71c73 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.757954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763d0b1-4f58-471a-ae0f-3f2c6c254e83 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.759469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9cd2c-912a-4f13-97ad-7a39b4f61706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.760574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9cd2c-912a-4f13-97ad-7a39b4f61706 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.761068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9cd2c-912a-4f13-97ad-7a39b4f61706 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.761577 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9cd2c-912a-4f13-97ad-7a39b4f61706 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.772758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9cd2c-912a-4f13-97ad-7a39b4f61706 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.774298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9cd2c-912a-4f13-97ad-7a39b4f61706 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7ee9cd2c-912a-4f13-97ad-7a39b4f61706 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:46.786810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:46.788240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 281/586] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:46.790672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9cd2c-912a-4f13-97ad-7a39b4f61706 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:46.808513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108741a-eb52-4ea5-b57e-93105a114bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.809169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108741a-eb52-4ea5-b57e-93105a114bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.809491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108741a-eb52-4ea5-b57e-93105a114bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.809782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108741a-eb52-4ea5-b57e-93105a114bd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.822846 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108741a-eb52-4ea5-b57e-93105a114bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.823944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108741a-eb52-4ea5-b57e-93105a114bd9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-d108741a-eb52-4ea5-b57e-93105a114bd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:46.844049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:46.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:46.847895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:46.847895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:46.847895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:46.847895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:46.847895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 306/587] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:46.847895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108741a-eb52-4ea5-b57e-93105a114bd9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:46.858258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:46.920685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9cd2c-912a-4f13-97ad-7a39b4f61706 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.946480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:46.958691 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108741a-eb52-4ea5-b57e-93105a114bd9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:46.962321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e205257-0b20-4581-95ee-7102062a8484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:46.962976 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e205257-0b20-4581-95ee-7102062a8484 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:46.963261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e205257-0b20-4581-95ee-7102062a8484 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:46.963540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e205257-0b20-4581-95ee-7102062a8484 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:46.974476 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e205257-0b20-4581-95ee-7102062a8484 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:46.975938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e205257-0b20-4581-95ee-7102062a8484 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4e205257-0b20-4581-95ee-7102062a8484 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:46.991034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:46.991961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:46.992847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:46.992847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:46.992847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:46.992847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:46.992847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 282/588] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:46.993137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e205257-0b20-4581-95ee-7102062a8484 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:47.001323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0bad8-7321-41fb-aea0-2a232127431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:47.002408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0bad8-7321-41fb-aea0-2a232127431e None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:47.002806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0bad8-7321-41fb-aea0-2a232127431e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:47.003092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0bad8-7321-41fb-aea0-2a232127431e None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/5706aa2d6bf54f05badf9588f51f4677/roles/31125b687d5e4a29b891ade41748f51e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:47.020282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0bad8-7321-41fb-aea0-2a232127431e None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=5706aa2d6bf54f05badf9588f51f4677, role_id=31125b687d5e4a29b891ade41748f51e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:47.021946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0bad8-7321-41fb-aea0-2a232127431e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:47.098217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e205257-0b20-4581-95ee-7102062a8484 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:47.114580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 307/589] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:09:46 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/5706aa2d6bf54f05badf9588f51f4677/roles/31125b687d5e4a29b891ade41748f51e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:47.117209 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0bad8-7321-41fb-aea0-2a232127431e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:47.163960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e2ac9-69e3-4893-ae03-8a0e403febe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:47.164608 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e2ac9-69e3-4893-ae03-8a0e403febe4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:47.164893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e2ac9-69e3-4893-ae03-8a0e403febe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:47.165169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e2ac9-69e3-4893-ae03-8a0e403febe4 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:47.166017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e2ac9-69e3-4893-ae03-8a0e403febe4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:47.166336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e2ac9-69e3-4893-ae03-8a0e403febe4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:47.167827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e2ac9-69e3-4893-ae03-8a0e403febe4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:47.193184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 283/590] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 28 06:09:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:47.194517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e2ac9-69e3-4893-ae03-8a0e403febe4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:47.223415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:47.236098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:47.273664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0bad8-7321-41fb-aea0-2a232127431e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:47.319764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bc6ab-d609-488a-8cf7-635c8a49ae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:47.320826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bc6ab-d609-488a-8cf7-635c8a49ae65 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:47.321481 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bc6ab-d609-488a-8cf7-635c8a49ae65 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:47.322097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bc6ab-d609-488a-8cf7-635c8a49ae65 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:47.323726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13bc6ab-d609-488a-8cf7-635c8a49ae65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:47.323726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13bc6ab-d609-488a-8cf7-635c8a49ae65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:47.324916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13bc6ab-d609-488a-8cf7-635c8a49ae65 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:47.329295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e2ac9-69e3-4893-ae03-8a0e403febe4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:47.340021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 308/591] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:47.341031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bc6ab-d609-488a-8cf7-635c8a49ae65 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:47.379281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc73479-4fe6-4a76-b30c-f59cb0df7f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:47.380540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc73479-4fe6-4a76-b30c-f59cb0df7f17 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:47.381036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc73479-4fe6-4a76-b30c-f59cb0df7f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:47.381555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc73479-4fe6-4a76-b30c-f59cb0df7f17 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:47.382741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc73479-4fe6-4a76-b30c-f59cb0df7f17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:47.384218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc73479-4fe6-4a76-b30c-f59cb0df7f17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:47.385849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc73479-4fe6-4a76-b30c-f59cb0df7f17 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:47.403485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 284/592] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:47.404645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc73479-4fe6-4a76-b30c-f59cb0df7f17 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:47.454838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bc6ab-d609-488a-8cf7-635c8a49ae65 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:47.490154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:47.506973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f613f-3571-4f9e-b915-9422409555e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:47.507634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f613f-3571-4f9e-b915-9422409555e3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:47.507924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f613f-3571-4f9e-b915-9422409555e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:47.508208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f613f-3571-4f9e-b915-9422409555e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:47.529332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f613f-3571-4f9e-b915-9422409555e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:47.529332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f613f-3571-4f9e-b915-9422409555e3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:47.540354 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-ef6f613f-3571-4f9e-b915-9422409555e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:47.545148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:47.546237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:47.547471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:47.547471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:47.547471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:47.547471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:47.549391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 309/593] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:47.550750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f613f-3571-4f9e-b915-9422409555e3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:47.555471 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc73479-4fe6-4a76-b30c-f59cb0df7f17 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:47.602510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6603a-f8ea-4e30-9d47-79d52e7f2d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:47.603538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6603a-f8ea-4e30-9d47-79d52e7f2d97 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:47.604020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6603a-f8ea-4e30-9d47-79d52e7f2d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:47.605026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6603a-f8ea-4e30-9d47-79d52e7f2d97 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:47.616924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6603a-f8ea-4e30-9d47-79d52e7f2d97 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:47.618740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6603a-f8ea-4e30-9d47-79d52e7f2d97 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d2c6603a-f8ea-4e30-9d47-79d52e7f2d97 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:47.633781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:47.634807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:47.635917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 285/594] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:47.636981 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6603a-f8ea-4e30-9d47-79d52e7f2d97 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:47.682632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f613f-3571-4f9e-b915-9422409555e3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:47.718664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4fb81-2b4d-47db-9198-44f61c108b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:47.719720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4fb81-2b4d-47db-9198-44f61c108b38 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:47.720213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4fb81-2b4d-47db-9198-44f61c108b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:47.720734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4fb81-2b4d-47db-9198-44f61c108b38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:47.728371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4fb81-2b4d-47db-9198-44f61c108b38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:47.730223 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4fb81-2b4d-47db-9198-44f61c108b38 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-b9e4fb81-2b4d-47db-9198-44f61c108b38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:47.742988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:47.743906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:47.744783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:47.744783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:47.744783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:47.744783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:47.744783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:47.744783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:47.744783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:47.744783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:47.751542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 310/595] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:47.752800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4fb81-2b4d-47db-9198-44f61c108b38 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:47.759536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6603a-f8ea-4e30-9d47-79d52e7f2d97 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:47.801216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ddb8e-b988-4a39-8903-1012cede6006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:47.801920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ddb8e-b988-4a39-8903-1012cede6006 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:47.802216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ddb8e-b988-4a39-8903-1012cede6006 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:47.802496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ddb8e-b988-4a39-8903-1012cede6006 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:47.803196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ddb8e-b988-4a39-8903-1012cede6006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:47.803505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ddb8e-b988-4a39-8903-1012cede6006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:47.805644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ddb8e-b988-4a39-8903-1012cede6006 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:47.821965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 286/596] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:47.823847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ddb8e-b988-4a39-8903-1012cede6006 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:47.888648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4fb81-2b4d-47db-9198-44f61c108b38 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:47.896895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:47.918581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:47.936427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ed75f-eadd-49eb-ac2f-063ea82256d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:47.937889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ed75f-eadd-49eb-ac2f-063ea82256d3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:47.938409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ed75f-eadd-49eb-ac2f-063ea82256d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:47.939175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ed75f-eadd-49eb-ac2f-063ea82256d3 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:47.940738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ed75f-eadd-49eb-ac2f-063ea82256d3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:47.941451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ed75f-eadd-49eb-ac2f-063ea82256d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:47.943819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ed75f-eadd-49eb-ac2f-063ea82256d3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:47.948125 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ddb8e-b988-4a39-8903-1012cede6006 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:47.969693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 311/597] 10.222.0.69 () {56 vars in 1139 bytes} [Thu Jul 28 06:09:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:47.971133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ed75f-eadd-49eb-ac2f-063ea82256d3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:47.996834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b73a6-3202-48ab-badb-373fe59c4dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:47.998475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b73a6-3202-48ab-badb-373fe59c4dcb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:47.999024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b73a6-3202-48ab-badb-373fe59c4dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:47.999808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b73a6-3202-48ab-badb-373fe59c4dcb None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:48.001108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b73a6-3202-48ab-badb-373fe59c4dcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:48.002103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b73a6-3202-48ab-badb-373fe59c4dcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:48.005012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b73a6-3202-48ab-badb-373fe59c4dcb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:48.022972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 287/598] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:48.024150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b73a6-3202-48ab-badb-373fe59c4dcb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:48.079513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:48.093667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ed75f-eadd-49eb-ac2f-063ea82256d3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:48.123235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:48.139969 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c757e-c00f-4820-8658-bff14146c8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:48.141117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c757e-c00f-4820-8658-bff14146c8da None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:48.141639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c757e-c00f-4820-8658-bff14146c8da None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:48.142118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c757e-c00f-4820-8658-bff14146c8da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:48.150249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c757e-c00f-4820-8658-bff14146c8da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:48.152003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c757e-c00f-4820-8658-bff14146c8da None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:48.156012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b73a6-3202-48ab-badb-373fe59c4dcb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-487c757e-c00f-4820-8658-bff14146c8da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:48.166275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:48.167391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:48.168252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:48.168252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:48.168252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:48.168252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:48.168252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:48.168252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:48.168252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:48.168252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:48.170848 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 312/599] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:48.171961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c757e-c00f-4820-8658-bff14146c8da None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:48.199802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed3512-cc1a-4ed7-abbd-a28c3b39767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:48.200491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed3512-cc1a-4ed7-abbd-a28c3b39767c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:48.200767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed3512-cc1a-4ed7-abbd-a28c3b39767c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:48.201048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed3512-cc1a-4ed7-abbd-a28c3b39767c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:48.212723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed3512-cc1a-4ed7-abbd-a28c3b39767c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:48.214348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed3512-cc1a-4ed7-abbd-a28c3b39767c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-eaed3512-cc1a-4ed7-abbd-a28c3b39767c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:48.230031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:48.230961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:48.232444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:48.232444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:48.232444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:48.232444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:48.232444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 288/600] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:48.232727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed3512-cc1a-4ed7-abbd-a28c3b39767c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:48.295544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c757e-c00f-4820-8658-bff14146c8da None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:48.337721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32666e-d49a-47c6-8a92-0cb57c60376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:48.338924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32666e-d49a-47c6-8a92-0cb57c60376d None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:48.339530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32666e-d49a-47c6-8a92-0cb57c60376d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:48.340033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32666e-d49a-47c6-8a92-0cb57c60376d None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/cc4fe04b11ec412e8f4eb67c6e54b8f7/roles/31125b687d5e4a29b891ade41748f51e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:48.345884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed3512-cc1a-4ed7-abbd-a28c3b39767c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:48.358434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32666e-d49a-47c6-8a92-0cb57c60376d None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=cc4fe04b11ec412e8f4eb67c6e54b8f7, role_id=31125b687d5e4a29b891ade41748f51e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:48.360465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32666e-d49a-47c6-8a92-0cb57c60376d None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:48.372631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:48.387154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4250b8-8106-4d20-865e-1654edf1638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:48.388198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4250b8-8106-4d20-865e-1654edf1638c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:48.388554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4250b8-8106-4d20-865e-1654edf1638c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:48.388909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4250b8-8106-4d20-865e-1654edf1638c None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:48.389788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4250b8-8106-4d20-865e-1654edf1638c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:48.390169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4250b8-8106-4d20-865e-1654edf1638c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:48.391706 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4250b8-8106-4d20-865e-1654edf1638c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:48.405539 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 289/601] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:48.407257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4250b8-8106-4d20-865e-1654edf1638c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:48.431034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:48.445482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 313/602] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:09:48 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/cc4fe04b11ec412e8f4eb67c6e54b8f7/roles/31125b687d5e4a29b891ade41748f51e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:48.447169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32666e-d49a-47c6-8a92-0cb57c60376d None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:48.528223 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4250b8-8106-4d20-865e-1654edf1638c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:48.569181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b50d37-6ae8-4324-9200-4da3c5c22b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:48.569776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b50d37-6ae8-4324-9200-4da3c5c22b39 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:48.570064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b50d37-6ae8-4324-9200-4da3c5c22b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:48.570348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b50d37-6ae8-4324-9200-4da3c5c22b39 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:48.571077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b50d37-6ae8-4324-9200-4da3c5c22b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:48.571392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b50d37-6ae8-4324-9200-4da3c5c22b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:48.572840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b50d37-6ae8-4324-9200-4da3c5c22b39 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:48.586757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 290/603] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:48.588209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0fec0-e65e-487b-8d5b-67e9f37b8ee1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:48.588514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0fec0-e65e-487b-8d5b-67e9f37b8ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:48.588797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0fec0-e65e-487b-8d5b-67e9f37b8ee1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:48.590372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 291/604] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:48.594205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32666e-d49a-47c6-8a92-0cb57c60376d None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:48.595251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0fec0-e65e-487b-8d5b-67e9f37b8ee1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:48.636991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f070b45-e5f0-4856-9b5f-b6a5121b3fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:48.638297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f070b45-e5f0-4856-9b5f-b6a5121b3fb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:48.638590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f070b45-e5f0-4856-9b5f-b6a5121b3fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:48.638874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f070b45-e5f0-4856-9b5f-b6a5121b3fb6 None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/cbbbf34c238d44db8588d56f35827390/roles/31125b687d5e4a29b891ade41748f51e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:48.648549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:48.659214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f070b45-e5f0-4856-9b5f-b6a5121b3fb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=cbbbf34c238d44db8588d56f35827390, role_id=31125b687d5e4a29b891ade41748f51e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:48.660983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f070b45-e5f0-4856-9b5f-b6a5121b3fb6 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:48.710442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0fec0-e65e-487b-8d5b-67e9f37b8ee1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:48.746768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458dcd0-5819-416a-a3e6-56749271c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:48.747798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458dcd0-5819-416a-a3e6-56749271c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:48.748260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458dcd0-5819-416a-a3e6-56749271c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:48.748695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458dcd0-5819-416a-a3e6-56749271c5e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:48.756510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458dcd0-5819-416a-a3e6-56749271c5e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:48.758265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458dcd0-5819-416a-a3e6-56749271c5e6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8458dcd0-5819-416a-a3e6-56749271c5e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:48.771347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:48.772335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:48.773235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:48.773235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:48.773235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:48.773235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:48.773235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:48.773235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:48.773235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:48.773235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:48.773235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 292/605] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:48.774691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b5584-c6f6-499e-8647-b210427bea65 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:48.775027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b5584-c6f6-499e-8647-b210427bea65 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:48.775317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b5584-c6f6-499e-8647-b210427bea65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:48.794738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ca8b5584-c6f6-499e-8647-b210427bea65 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:48.801337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ca8b5584-c6f6-499e-8647-b210427bea65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:48.801930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 314/606] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:09:48 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/cbbbf34c238d44db8588d56f35827390/roles/31125b687d5e4a29b891ade41748f51e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:48.804013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f070b45-e5f0-4856-9b5f-b6a5121b3fb6 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:48.894469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 293/607] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:48.919997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b5584-c6f6-499e-8647-b210427bea65 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:48.920750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:48.974709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:49.041632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f070b45-e5f0-4856-9b5f-b6a5121b3fb6 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:49.061188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b5584-c6f6-499e-8647-b210427bea65 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:49.079291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f87d43-f55b-4705-992b-2e263632d36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:49.079940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f87d43-f55b-4705-992b-2e263632d36d None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:49.080241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f87d43-f55b-4705-992b-2e263632d36d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:49.081117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f87d43-f55b-4705-992b-2e263632d36d None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:49.082612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f87d43-f55b-4705-992b-2e263632d36d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:49.082936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f87d43-f55b-4705-992b-2e263632d36d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:49.084411 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f87d43-f55b-4705-992b-2e263632d36d None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:49.097398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 315/608] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:49.100486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce694aef-5713-4668-a1e4-715d6580ad83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:49.101154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce694aef-5713-4668-a1e4-715d6580ad83 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:49.101475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce694aef-5713-4668-a1e4-715d6580ad83 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:49.101762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce694aef-5713-4668-a1e4-715d6580ad83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:49.103201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f87d43-f55b-4705-992b-2e263632d36d None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:49.119767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce694aef-5713-4668-a1e4-715d6580ad83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ce694aef-5713-4668-a1e4-715d6580ad83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:49.126614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:49.128173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:49.129089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 294/609] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:49.134101 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce694aef-5713-4668-a1e4-715d6580ad83 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:49.233630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f87d43-f55b-4705-992b-2e263632d36d None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:49.243611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:49.249673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:49.259613 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce694aef-5713-4668-a1e4-715d6580ad83 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:49.278128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917966a-efd2-44fd-8dc4-023c20ae0f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:49.279729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917966a-efd2-44fd-8dc4-023c20ae0f95 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:49.280272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917966a-efd2-44fd-8dc4-023c20ae0f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:49.280759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917966a-efd2-44fd-8dc4-023c20ae0f95 None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/05574eace38f4b6d945c7b1927fdadd8/roles/31125b687d5e4a29b891ade41748f51e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:49.298289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917966a-efd2-44fd-8dc4-023c20ae0f95 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=05574eace38f4b6d945c7b1927fdadd8, role_id=31125b687d5e4a29b891ade41748f51e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:49.301416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917966a-efd2-44fd-8dc4-023c20ae0f95 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:49.326499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727677e-e077-49a9-a09e-babb1fe8090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:49.327129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727677e-e077-49a9-a09e-babb1fe8090f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:49.327419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727677e-e077-49a9-a09e-babb1fe8090f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:49.327699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727677e-e077-49a9-a09e-babb1fe8090f None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:49.328472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727677e-e077-49a9-a09e-babb1fe8090f None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:49.328785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727677e-e077-49a9-a09e-babb1fe8090f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:49.330298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727677e-e077-49a9-a09e-babb1fe8090f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:49.340751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 295/610] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:49.349365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727677e-e077-49a9-a09e-babb1fe8090f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:49.379963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 316/611] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:09:49 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/05574eace38f4b6d945c7b1927fdadd8/roles/31125b687d5e4a29b891ade41748f51e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:09:49.403592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81a630-e046-439f-bfef-3fb7dbccdc79 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:49.404280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81a630-e046-439f-bfef-3fb7dbccdc79 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:49.404852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81a630-e046-439f-bfef-3fb7dbccdc79 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:49.406909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 317/612] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:49.419659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a26e4f-44f0-4ecb-a536-6f369352ac88 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:49.420324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a26e4f-44f0-4ecb-a536-6f369352ac88 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:49.421023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a26e4f-44f0-4ecb-a536-6f369352ac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:49.465787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-d2a26e4f-44f0-4ecb-a536-6f369352ac88 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:49.467035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d2a26e4f-44f0-4ecb-a536-6f369352ac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:49.527684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727677e-e077-49a9-a09e-babb1fe8090f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:49.568895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea823db-88c0-4971-ac36-973c14d50f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:49.569616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea823db-88c0-4971-ac36-973c14d50f76 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:49.569899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea823db-88c0-4971-ac36-973c14d50f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:49.570174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea823db-88c0-4971-ac36-973c14d50f76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:49.582573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea823db-88c0-4971-ac36-973c14d50f76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:49.584041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea823db-88c0-4971-ac36-973c14d50f76 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7ea823db-88c0-4971-ac36-973c14d50f76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:49.598912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:49.599872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:49.601202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:49.601202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:49.601202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:49.601202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:49.601202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 296/613] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:49.605840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea823db-88c0-4971-ac36-973c14d50f76 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:49.618985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:49.631418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 318/614] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:09:49.679213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a26e4f-44f0-4ecb-a536-6f369352ac88 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:49.741024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea823db-88c0-4971-ac36-973c14d50f76 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:49.812135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b845fad-72f1-425d-9843-893f044f36f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:49.813120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b845fad-72f1-425d-9843-893f044f36f9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:49.815613 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b845fad-72f1-425d-9843-893f044f36f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:49.816158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b845fad-72f1-425d-9843-893f044f36f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:49.817433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b845fad-72f1-425d-9843-893f044f36f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:49.819213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b845fad-72f1-425d-9843-893f044f36f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:49.820843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b845fad-72f1-425d-9843-893f044f36f9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:49.849845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 297/615] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:49.856164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b845fad-72f1-425d-9843-893f044f36f9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:49.859651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:49.888264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a26e4f-44f0-4ecb-a536-6f369352ac88 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:49.958104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35da20-f27b-4540-ac1f-332e8699337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:49.958771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35da20-f27b-4540-ac1f-332e8699337f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:49.959058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35da20-f27b-4540-ac1f-332e8699337f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:49.959337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35da20-f27b-4540-ac1f-332e8699337f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:49.982512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35da20-f27b-4540-ac1f-332e8699337f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-fe35da20-f27b-4540-ac1f-332e8699337f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:49.988478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:49.989464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:49.990703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 319/616] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:50.004649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35da20-f27b-4540-ac1f-332e8699337f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:50.005440 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:50.063573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b845fad-72f1-425d-9843-893f044f36f9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:50.100421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709c4ff-7cc5-449e-a186-c607aadde7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:50.101073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709c4ff-7cc5-449e-a186-c607aadde7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:50.101384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709c4ff-7cc5-449e-a186-c607aadde7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:50.101663 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709c4ff-7cc5-449e-a186-c607aadde7b4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:50.109295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709c4ff-7cc5-449e-a186-c607aadde7b4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:50.110814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709c4ff-7cc5-449e-a186-c607aadde7b4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-5709c4ff-7cc5-449e-a186-c607aadde7b4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:50.123334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:50.124286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:50.125920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 298/617] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:09:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:50.130483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709c4ff-7cc5-449e-a186-c607aadde7b4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:50.191977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35da20-f27b-4540-ac1f-332e8699337f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:50.240095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709c4ff-7cc5-449e-a186-c607aadde7b4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:50.266617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2de76-2289-4dac-ad3d-613de527114b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:50.267280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2de76-2289-4dac-ad3d-613de527114b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:50.267564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2de76-2289-4dac-ad3d-613de527114b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:50.267839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2de76-2289-4dac-ad3d-613de527114b None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:50.269290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2de76-2289-4dac-ad3d-613de527114b None admin] RBAC: Adding query filter params (name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:50.269290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2de76-2289-4dac-ad3d-613de527114b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:50.270435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2de76-2289-4dac-ad3d-613de527114b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:50.276429 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:50.279854 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70741006-3680-409f-8f0a-6ed0539547fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:50.280482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70741006-3680-409f-8f0a-6ed0539547fe None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:50.280775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70741006-3680-409f-8f0a-6ed0539547fe None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:50.281055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70741006-3680-409f-8f0a-6ed0539547fe None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:50.281899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70741006-3680-409f-8f0a-6ed0539547fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:50.282212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70741006-3680-409f-8f0a-6ed0539547fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:50.284534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70741006-3680-409f-8f0a-6ed0539547fe None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:50.288983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 320/618] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:50.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2de76-2289-4dac-ad3d-613de527114b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:50.314008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:50.322189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 299/619] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 28 06:09:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:50.330782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70741006-3680-409f-8f0a-6ed0539547fe None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:50.404407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2de76-2289-4dac-ad3d-613de527114b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:50.440372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ae2f-f7f4-4a60-8d52-81dda90a18f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:50.441202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ae2f-f7f4-4a60-8d52-81dda90a18f0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:50.441599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ae2f-f7f4-4a60-8d52-81dda90a18f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:50.441971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ae2f-f7f4-4a60-8d52-81dda90a18f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:50.461123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473ae2f-f7f4-4a60-8d52-81dda90a18f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:50.462619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473ae2f-f7f4-4a60-8d52-81dda90a18f0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:50.498250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70741006-3680-409f-8f0a-6ed0539547fe None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-7473ae2f-f7f4-4a60-8d52-81dda90a18f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:50.502276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:50.503162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:50.504489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:50.504489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:50.504489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:50.504489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:50.504489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 321/620] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:50.518866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ae2f-f7f4-4a60-8d52-81dda90a18f0 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:50.544441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac287410-de67-46cf-af69-8d66086f6947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:50.545101 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac287410-de67-46cf-af69-8d66086f6947 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:50.545455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac287410-de67-46cf-af69-8d66086f6947 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:50.545801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac287410-de67-46cf-af69-8d66086f6947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:50.557020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac287410-de67-46cf-af69-8d66086f6947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:50.558551 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac287410-de67-46cf-af69-8d66086f6947 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ac287410-de67-46cf-af69-8d66086f6947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:50.574445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:50.575379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:50.576723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:50.576723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:50.576723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:50.576723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:50.576723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 300/621] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:50.582292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac287410-de67-46cf-af69-8d66086f6947 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:50.632989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:50.633830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:50.689868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac287410-de67-46cf-af69-8d66086f6947 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:50.706658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ae2f-f7f4-4a60-8d52-81dda90a18f0 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:50.729294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64237f-2e2f-4cd1-99c8-7ce5674f94eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:50.730894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64237f-2e2f-4cd1-99c8-7ce5674f94eb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:50.731507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64237f-2e2f-4cd1-99c8-7ce5674f94eb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:50.733233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64237f-2e2f-4cd1-99c8-7ce5674f94eb None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:50.734170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64237f-2e2f-4cd1-99c8-7ce5674f94eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:50.734765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64237f-2e2f-4cd1-99c8-7ce5674f94eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:50.738738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64237f-2e2f-4cd1-99c8-7ce5674f94eb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:50.759649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 301/622] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:09:50.765770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64237f-2e2f-4cd1-99c8-7ce5674f94eb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:50.777342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a76e5-0052-4c05-8e4b-755e06f014d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:50.778381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a76e5-0052-4c05-8e4b-755e06f014d5 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:50.778381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a76e5-0052-4c05-8e4b-755e06f014d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:50.778685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a76e5-0052-4c05-8e4b-755e06f014d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:50.779404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a76e5-0052-4c05-8e4b-755e06f014d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:50.779716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a76e5-0052-4c05-8e4b-755e06f014d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:50.781239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a76e5-0052-4c05-8e4b-755e06f014d5 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:50.813353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 322/623] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:50.818963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a76e5-0052-4c05-8e4b-755e06f014d5 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:50.921603 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:50.926533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64237f-2e2f-4cd1-99c8-7ce5674f94eb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:50.977030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06c391-366c-4e32-b511-f2e18254941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:50.977726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06c391-366c-4e32-b511-f2e18254941a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:50.978014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06c391-366c-4e32-b511-f2e18254941a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:50.978297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06c391-366c-4e32-b511-f2e18254941a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:50.994332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06c391-366c-4e32-b511-f2e18254941a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:50.995843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06c391-366c-4e32-b511-f2e18254941a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-0e06c391-366c-4e32-b511-f2e18254941a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:51.010455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:51.012022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:51.013207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:51.013207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:51.013207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:51.013207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:51.013207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:51.013207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:51.013207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:51.013207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:51.020586 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 302/624] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:51.022587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7cbb2-b1c5-48bb-9b3d-4b0a9b833475 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.023114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7cbb2-b1c5-48bb-9b3d-4b0a9b833475 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.023713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7cbb2-b1c5-48bb-9b3d-4b0a9b833475 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.025630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 303/625] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:51.030572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a76e5-0052-4c05-8e4b-755e06f014d5 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:51.031613 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7cbb2-b1c5-48bb-9b3d-4b0a9b833475 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:51.058002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:51.076401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35777507-1763-46fb-9e6c-6ee533e54664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:51.077502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35777507-1763-46fb-9e6c-6ee533e54664 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.077998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35777507-1763-46fb-9e6c-6ee533e54664 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.078529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35777507-1763-46fb-9e6c-6ee533e54664 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.091310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35777507-1763-46fb-9e6c-6ee533e54664 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:51.093600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35777507-1763-46fb-9e6c-6ee533e54664 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-35777507-1763-46fb-9e6c-6ee533e54664 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:51.110731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:51.112343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:51.113781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:51.113781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:51.113781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:51.113781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:51.113781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:51.113781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:51.113781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:51.113781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:51.113781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:09:51.113781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:51.119778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 323/626] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 28 06:09:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:51.119778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe473d-4d84-436d-8aa4-96704907b66c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.119778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe473d-4d84-436d-8aa4-96704907b66c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.119778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe473d-4d84-436d-8aa4-96704907b66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.138081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-7afe473d-4d84-436d-8aa4-96704907b66c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:51.139564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7afe473d-4d84-436d-8aa4-96704907b66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:51.167019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7cbb2-b1c5-48bb-9b3d-4b0a9b833475 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:51.207312 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b94852-e61e-4c98-bb6f-6d88d5e45cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:51.207844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b94852-e61e-4c98-bb6f-6d88d5e45cea None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.208127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b94852-e61e-4c98-bb6f-6d88d5e45cea None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.208403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b94852-e61e-4c98-bb6f-6d88d5e45cea None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.209183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b94852-e61e-4c98-bb6f-6d88d5e45cea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:51.209545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b94852-e61e-4c98-bb6f-6d88d5e45cea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:51.211023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b94852-e61e-4c98-bb6f-6d88d5e45cea None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:51.225480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 304/627] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:51.226413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b94852-e61e-4c98-bb6f-6d88d5e45cea None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:51.242141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 324/628] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:51.244639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed71b9-9f62-4d97-9425-557d6830f1a1 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.245136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed71b9-9f62-4d97-9425-557d6830f1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.245800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed71b9-9f62-4d97-9425-557d6830f1a1 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.248407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 325/629] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:51.249559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed71b9-9f62-4d97-9425-557d6830f1a1 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:51.255294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:51.339913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b94852-e61e-4c98-bb6f-6d88d5e45cea None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:51.375802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c54c0-3e73-463b-81b0-1f14bd76421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:51.376435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c54c0-3e73-463b-81b0-1f14bd76421e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.376718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c54c0-3e73-463b-81b0-1f14bd76421e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.377006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c54c0-3e73-463b-81b0-1f14bd76421e None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.378456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c54c0-3e73-463b-81b0-1f14bd76421e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:51.378960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c54c0-3e73-463b-81b0-1f14bd76421e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:51.380431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c54c0-3e73-463b-81b0-1f14bd76421e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:51.385175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed71b9-9f62-4d97-9425-557d6830f1a1 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:51.401136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 305/630] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 28 06:09:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:51.401978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:51.406875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab17e5d-3765-4f1c-a1f5-de88531ebb21 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.407157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab17e5d-3765-4f1c-a1f5-de88531ebb21 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.407442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab17e5d-3765-4f1c-a1f5-de88531ebb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.425743 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab5104-b802-4634-a93a-9b38817bea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:51.426410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4ab17e5d-3765-4f1c-a1f5-de88531ebb21 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:51.426729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5104-b802-4634-a93a-9b38817bea60 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.427111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5104-b802-4634-a93a-9b38817bea60 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.427486 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5104-b802-4634-a93a-9b38817bea60 None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/77571de30e934d6590ff7b58a000f5e2/roles/31125b687d5e4a29b891ade41748f51e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.427791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4ab17e5d-3765-4f1c-a1f5-de88531ebb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:51.447053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab5104-b802-4634-a93a-9b38817bea60 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=77571de30e934d6590ff7b58a000f5e2, role_id=31125b687d5e4a29b891ade41748f51e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:51.449993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab5104-b802-4634-a93a-9b38817bea60 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:51.500558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 306/631] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:51.501710 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab17e5d-3765-4f1c-a1f5-de88531ebb21 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:51.510873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:51.527080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 326/632] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:09:51 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/77571de30e934d6590ff7b58a000f5e2/roles/31125b687d5e4a29b891ade41748f51e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:09:51.528132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab5104-b802-4634-a93a-9b38817bea60 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:51.642177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab17e5d-3765-4f1c-a1f5-de88531ebb21 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:51.674660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab5104-b802-4634-a93a-9b38817bea60 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:51.688433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea3011-d57f-4111-9526-f537dccc1f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:51.689116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3011-d57f-4111-9526-f537dccc1f3a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.689554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3011-d57f-4111-9526-f537dccc1f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.690494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3011-d57f-4111-9526-f537dccc1f3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.704536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea3011-d57f-4111-9526-f537dccc1f3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7aea3011-d57f-4111-9526-f537dccc1f3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:51.710859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:51.711834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:51.712685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 307/633] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:51.713426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea3011-d57f-4111-9526-f537dccc1f3a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:51.723645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc043751-857c-4777-9cb1-9ccb75c4ab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:51.725087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc043751-857c-4777-9cb1-9ccb75c4ab3f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.725643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc043751-857c-4777-9cb1-9ccb75c4ab3f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.726130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc043751-857c-4777-9cb1-9ccb75c4ab3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.772843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:51.792164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:51.818182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc043751-857c-4777-9cb1-9ccb75c4ab3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:51.819942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc043751-857c-4777-9cb1-9ccb75c4ab3f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:51.838705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea3011-d57f-4111-9526-f537dccc1f3a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:51.886954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307b63d-4be0-4687-b862-9cbc46ca52f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:51.888134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307b63d-4be0-4687-b862-9cbc46ca52f6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:51.888520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307b63d-4be0-4687-b862-9cbc46ca52f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:51.888916 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307b63d-4be0-4687-b862-9cbc46ca52f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:51.901722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307b63d-4be0-4687-b862-9cbc46ca52f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4307b63d-4be0-4687-b862-9cbc46ca52f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:51.907171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:51.908325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:51.909742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:51.912098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 308/634] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:51.913162 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307b63d-4be0-4687-b862-9cbc46ca52f6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:51.938345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 327/635] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:51.939445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc043751-857c-4777-9cb1-9ccb75c4ab3f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.015588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:52.023175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307b63d-4be0-4687-b862-9cbc46ca52f6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:52.051101 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:52.060332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2d52a-20cb-4503-88cf-d14ff6b18a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:52.060952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2d52a-20cb-4503-88cf-d14ff6b18a98 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.061232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2d52a-20cb-4503-88cf-d14ff6b18a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.061537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2d52a-20cb-4503-88cf-d14ff6b18a98 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.062263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2d52a-20cb-4503-88cf-d14ff6b18a98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:52.062605 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2d52a-20cb-4503-88cf-d14ff6b18a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:52.063938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2d52a-20cb-4503-88cf-d14ff6b18a98 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:52.066334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc043751-857c-4777-9cb1-9ccb75c4ab3f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:52.077064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 309/636] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:52.078450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaac835-75b8-4bf8-ac1e-165aff8c7e19 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.078761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaac835-75b8-4bf8-ac1e-165aff8c7e19 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.079059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaac835-75b8-4bf8-ac1e-165aff8c7e19 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.085103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 310/637] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:52.085943 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaac835-75b8-4bf8-ac1e-165aff8c7e19 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.103751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c183a2-aec3-4a71-8c5e-1175ed8225a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:52.104401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c183a2-aec3-4a71-8c5e-1175ed8225a9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.104689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c183a2-aec3-4a71-8c5e-1175ed8225a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.104971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c183a2-aec3-4a71-8c5e-1175ed8225a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.105891 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c183a2-aec3-4a71-8c5e-1175ed8225a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:52.106200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c183a2-aec3-4a71-8c5e-1175ed8225a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:52.107579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c183a2-aec3-4a71-8c5e-1175ed8225a9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:52.121521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 328/638] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:52.121521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c183a2-aec3-4a71-8c5e-1175ed8225a9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.206760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaac835-75b8-4bf8-ac1e-165aff8c7e19 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:52.230497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c183a2-aec3-4a71-8c5e-1175ed8225a9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:52.245844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae2df8-5b96-4fee-91ba-8e0f4d21d30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:52.246716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae2df8-5b96-4fee-91ba-8e0f4d21d30d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.247000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae2df8-5b96-4fee-91ba-8e0f4d21d30d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.247275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae2df8-5b96-4fee-91ba-8e0f4d21d30d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.255022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:52.260044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae2df8-5b96-4fee-91ba-8e0f4d21d30d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:52.261554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae2df8-5b96-4fee-91ba-8e0f4d21d30d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:52.267181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85d3c0-0eed-4c6b-b3ec-5ac84ede82a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:52.268544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85d3c0-0eed-4c6b-b3ec-5ac84ede82a7 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.268895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85d3c0-0eed-4c6b-b3ec-5ac84ede82a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.269100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85d3c0-0eed-4c6b-b3ec-5ac84ede82a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c9ae2df8-5b96-4fee-91ba-8e0f4d21d30d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:52.277748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:52.278775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:52.279725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:52.279725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:52.279725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:52.279725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:52.279725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 311/639] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:52.281062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d6285-e883-4a93-ada7-ce314836c452 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.281062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d6285-e883-4a93-ada7-ce314836c452 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.281062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d6285-e883-4a93-ada7-ce314836c452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.287576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad85d3c0-0eed-4c6b-b3ec-5ac84ede82a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:52.289201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad85d3c0-0eed-4c6b-b3ec-5ac84ede82a7 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:52.302568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fd8d6285-e883-4a93-ada7-ce314836c452 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:52.303780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fd8d6285-e883-4a93-ada7-ce314836c452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-ad85d3c0-0eed-4c6b-b3ec-5ac84ede82a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:52.306948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:52.307853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:52.308741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:52.308741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:52.308741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:52.308741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:52.312594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 329/640] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:52.313550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85d3c0-0eed-4c6b-b3ec-5ac84ede82a7 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.343318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:52.383054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 312/641] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:52.393546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d6285-e883-4a93-ada7-ce314836c452 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.443153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85d3c0-0eed-4c6b-b3ec-5ac84ede82a7 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:52.484667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ddaa3-0059-46f7-8649-a9ce198201b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:52.485336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ddaa3-0059-46f7-8649-a9ce198201b6 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.485624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ddaa3-0059-46f7-8649-a9ce198201b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.485897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ddaa3-0059-46f7-8649-a9ce198201b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.493221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d6285-e883-4a93-ada7-ce314836c452 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:52.497109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ddaa3-0059-46f7-8649-a9ce198201b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:52.498648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ddaa3-0059-46f7-8649-a9ce198201b6 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-be7ddaa3-0059-46f7-8649-a9ce198201b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:52.514341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:52.515343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:52.516237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:52.516237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:52.516237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:52.516237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:52.516237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 330/642] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:52.516533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ddaa3-0059-46f7-8649-a9ce198201b6 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.535349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803f60a-5003-4083-a59b-d9c4d3b65d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:52.535895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803f60a-5003-4083-a59b-d9c4d3b65d8f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.536181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803f60a-5003-4083-a59b-d9c4d3b65d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.536456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803f60a-5003-4083-a59b-d9c4d3b65d8f None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.537156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803f60a-5003-4083-a59b-d9c4d3b65d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:52.537505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803f60a-5003-4083-a59b-d9c4d3b65d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:52.538896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803f60a-5003-4083-a59b-d9c4d3b65d8f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:52.546942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:52.555980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 313/643] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:52.555980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803f60a-5003-4083-a59b-d9c4d3b65d8f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.585083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:52.628071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ddaa3-0059-46f7-8649-a9ce198201b6 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:52.663354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dc045-e0ee-47aa-bc11-c83dc31dbc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:52.663988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dc045-e0ee-47aa-bc11-c83dc31dbc68 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.664280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dc045-e0ee-47aa-bc11-c83dc31dbc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.664563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dc045-e0ee-47aa-bc11-c83dc31dbc68 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.665264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dc045-e0ee-47aa-bc11-c83dc31dbc68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:52.665607 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dc045-e0ee-47aa-bc11-c83dc31dbc68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:52.666993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dc045-e0ee-47aa-bc11-c83dc31dbc68 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:52.671508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803f60a-5003-4083-a59b-d9c4d3b65d8f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:52.680678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 331/644] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:52.681959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dc045-e0ee-47aa-bc11-c83dc31dbc68 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.710550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387dc61-2aee-421c-8138-9912b7552f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:52.711540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387dc61-2aee-421c-8138-9912b7552f33 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.712125 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387dc61-2aee-421c-8138-9912b7552f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.712707 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387dc61-2aee-421c-8138-9912b7552f33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.725216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387dc61-2aee-421c-8138-9912b7552f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c387dc61-2aee-421c-8138-9912b7552f33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:52.731202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:52.732360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:52.733505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:52.738838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 314/645] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:52.739544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387dc61-2aee-421c-8138-9912b7552f33 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.788827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:52.803515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dc045-e0ee-47aa-bc11-c83dc31dbc68 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:52.834311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:52.838832 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6243ee-781b-4a25-a202-f96e93d5d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:52.839458 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6243ee-781b-4a25-a202-f96e93d5d316 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.839748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6243ee-781b-4a25-a202-f96e93d5d316 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.840021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6243ee-781b-4a25-a202-f96e93d5d316 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.840830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6243ee-781b-4a25-a202-f96e93d5d316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:52.841123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6243ee-781b-4a25-a202-f96e93d5d316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:52.842441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6243ee-781b-4a25-a202-f96e93d5d316 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:52.844680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387dc61-2aee-421c-8138-9912b7552f33 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:52.854377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 332/646] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:52.855216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6243ee-781b-4a25-a202-f96e93d5d316 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.888285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f2a42-f650-4485-bb35-0ed4ecb2c2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:52.888935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f2a42-f650-4485-bb35-0ed4ecb2c2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:52.889221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f2a42-f650-4485-bb35-0ed4ecb2c2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:52.889531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f2a42-f650-4485-bb35-0ed4ecb2c2e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:52.897889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f2a42-f650-4485-bb35-0ed4ecb2c2e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:52.899354 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f2a42-f650-4485-bb35-0ed4ecb2c2e7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c59f2a42-f650-4485-bb35-0ed4ecb2c2e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:52.912619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:52.913764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:52.914685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:52.914685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:52.914685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:52.914685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:52.914685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:52.914685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:52.914685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:52.914685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:52.914685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 315/647] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:52.914685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f2a42-f650-4485-bb35-0ed4ecb2c2e7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:52.976542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6243ee-781b-4a25-a202-f96e93d5d316 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.013880 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91efe96-5794-4983-b804-2e58b3e59d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.014538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91efe96-5794-4983-b804-2e58b3e59d9a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.014824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91efe96-5794-4983-b804-2e58b3e59d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.015109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91efe96-5794-4983-b804-2e58b3e59d9a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.020974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f2a42-f650-4485-bb35-0ed4ecb2c2e7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.024597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91efe96-5794-4983-b804-2e58b3e59d9a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:53.026124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91efe96-5794-4983-b804-2e58b3e59d9a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-e91efe96-5794-4983-b804-2e58b3e59d9a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:53.039590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:53.040715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:53.041696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 333/648] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:53.042305 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91efe96-5794-4983-b804-2e58b3e59d9a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:53.058299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:53.061113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b80f48-ed5d-466f-9bd3-1b72d57348f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.061795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80f48-ed5d-466f-9bd3-1b72d57348f7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.062084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80f48-ed5d-466f-9bd3-1b72d57348f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.062467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80f48-ed5d-466f-9bd3-1b72d57348f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.063394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b80f48-ed5d-466f-9bd3-1b72d57348f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:53.063534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b80f48-ed5d-466f-9bd3-1b72d57348f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:53.065080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b80f48-ed5d-466f-9bd3-1b72d57348f7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:53.073643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 316/649] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:53.074463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b80f48-ed5d-466f-9bd3-1b72d57348f7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:53.091812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:53.165758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91efe96-5794-4983-b804-2e58b3e59d9a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.184895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b80f48-ed5d-466f-9bd3-1b72d57348f7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.209650 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14721a2f-0d45-4765-b0be-f625daecfbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.210299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14721a2f-0d45-4765-b0be-f625daecfbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.210586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14721a2f-0d45-4765-b0be-f625daecfbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.210862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14721a2f-0d45-4765-b0be-f625daecfbe8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.219505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14721a2f-0d45-4765-b0be-f625daecfbe8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:53.221070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14721a2f-0d45-4765-b0be-f625daecfbe8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:53.222138 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c98efb-34df-4981-afa8-5b9352cbd953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.222765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c98efb-34df-4981-afa8-5b9352cbd953 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.223056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c98efb-34df-4981-afa8-5b9352cbd953 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.223328 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c98efb-34df-4981-afa8-5b9352cbd953 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.224080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c98efb-34df-4981-afa8-5b9352cbd953 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:53.224388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c98efb-34df-4981-afa8-5b9352cbd953 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:53.225843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c98efb-34df-4981-afa8-5b9352cbd953 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-14721a2f-0d45-4765-b0be-f625daecfbe8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:53.233455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:53.234567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 334/650] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:53.235512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14721a2f-0d45-4765-b0be-f625daecfbe8 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:53.249177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 317/651] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:53.249177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c98efb-34df-4981-afa8-5b9352cbd953 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:53.315187 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:53.333639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:53.357866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14721a2f-0d45-4765-b0be-f625daecfbe8 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.366795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c98efb-34df-4981-afa8-5b9352cbd953 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.398872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411a544-83e3-4afd-9884-1cf1303e4fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.399508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a544-83e3-4afd-9884-1cf1303e4fec None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.399795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a544-83e3-4afd-9884-1cf1303e4fec None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.400071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a544-83e3-4afd-9884-1cf1303e4fec None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.400876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411a544-83e3-4afd-9884-1cf1303e4fec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:53.401196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411a544-83e3-4afd-9884-1cf1303e4fec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:53.402679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411a544-83e3-4afd-9884-1cf1303e4fec None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:53.408874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599da9b2-63cb-4e7c-affa-6fa97a338423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.409899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599da9b2-63cb-4e7c-affa-6fa97a338423 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.410643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599da9b2-63cb-4e7c-affa-6fa97a338423 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.411090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599da9b2-63cb-4e7c-affa-6fa97a338423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.425496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 335/652] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 28 06:09:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:53.425496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411a544-83e3-4afd-9884-1cf1303e4fec None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:53.427129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599da9b2-63cb-4e7c-affa-6fa97a338423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:53.428597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599da9b2-63cb-4e7c-affa-6fa97a338423 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-599da9b2-63cb-4e7c-affa-6fa97a338423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:53.444154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:53.445163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:53.446110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:53.446110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:53.446110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:53.446110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:53.446110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 318/653] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:53.446110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599da9b2-63cb-4e7c-affa-6fa97a338423 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:53.542874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411a544-83e3-4afd-9884-1cf1303e4fec None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.552602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599da9b2-63cb-4e7c-affa-6fa97a338423 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.572160 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:53.576486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:53.582007 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc409f1-be1b-4749-b83f-bb4cef100130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.582636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc409f1-be1b-4749-b83f-bb4cef100130 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.584906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc409f1-be1b-4749-b83f-bb4cef100130 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.584906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc409f1-be1b-4749-b83f-bb4cef100130 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.584906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc409f1-be1b-4749-b83f-bb4cef100130 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:53.584906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc409f1-be1b-4749-b83f-bb4cef100130 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:53.585911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc409f1-be1b-4749-b83f-bb4cef100130 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:53.588168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc8dd3-aeb3-4855-89a8-c0090116b3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.589603 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc8dd3-aeb3-4855-89a8-c0090116b3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.589897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc8dd3-aeb3-4855-89a8-c0090116b3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.590183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc8dd3-aeb3-4855-89a8-c0090116b3a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.590990 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc8dd3-aeb3-4855-89a8-c0090116b3a9 None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=5706aa2d6bf54f05badf9588f51f4677) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:53.591311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc8dd3-aeb3-4855-89a8-c0090116b3a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:53.592778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc8dd3-aeb3-4855-89a8-c0090116b3a9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:53.609672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 319/654] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:53 2022] GET /identity/v3/role_assignments?user.id=5706aa2d6bf54f05badf9588f51f4677&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:53.609672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc8dd3-aeb3-4855-89a8-c0090116b3a9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:53.618851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 336/655] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 28 06:09:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:53.619776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc409f1-be1b-4749-b83f-bb4cef100130 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:53.732333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc8dd3-aeb3-4855-89a8-c0090116b3a9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.740182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc409f1-be1b-4749-b83f-bb4cef100130 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.777823 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3376a4-5c55-4476-8fd6-d79e229d7c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.778459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3376a4-5c55-4476-8fd6-d79e229d7c3b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.778738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3376a4-5c55-4476-8fd6-d79e229d7c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.779022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3376a4-5c55-4476-8fd6-d79e229d7c3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.780763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a484300-5acd-4cd2-aa98-087b0e5192f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.781421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a484300-5acd-4cd2-aa98-087b0e5192f4 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.781741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a484300-5acd-4cd2-aa98-087b0e5192f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.781993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a484300-5acd-4cd2-aa98-087b0e5192f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.782704 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a484300-5acd-4cd2-aa98-087b0e5192f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:53.783019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a484300-5acd-4cd2-aa98-087b0e5192f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:53.784465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a484300-5acd-4cd2-aa98-087b0e5192f4 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:53.796485 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 337/656] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:53.798608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5114fa-c6eb-4f89-ad1c-af2e02ba7a3b None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.799781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5114fa-c6eb-4f89-ad1c-af2e02ba7a3b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.800217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5114fa-c6eb-4f89-ad1c-af2e02ba7a3b None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.801725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 338/657] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:53.802447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5114fa-c6eb-4f89-ad1c-af2e02ba7a3b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:53.852850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:53.858057 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:53.866762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3376a4-5c55-4476-8fd6-d79e229d7c3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:53.868499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3376a4-5c55-4476-8fd6-d79e229d7c3b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:53.927319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5114fa-c6eb-4f89-ad1c-af2e02ba7a3b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:53.964497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2275d-3586-409f-94d5-d086da24bf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:53.965495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2275d-3586-409f-94d5-d086da24bf8b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:53.965981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2275d-3586-409f-94d5-d086da24bf8b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:53.966334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2275d-3586-409f-94d5-d086da24bf8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:53.991284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 320/658] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:53.991284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3376a4-5c55-4476-8fd6-d79e229d7c3b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:54.062908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2275d-3586-409f-94d5-d086da24bf8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:54.064529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2275d-3586-409f-94d5-d086da24bf8b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:54.082259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:54.094632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3376a4-5c55-4476-8fd6-d79e229d7c3b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:54.120605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:54.135541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d056e4-730c-4ccf-9eb2-1af50eb19cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:54.136362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d056e4-730c-4ccf-9eb2-1af50eb19cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:54.136648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d056e4-730c-4ccf-9eb2-1af50eb19cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:54.136927 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d056e4-730c-4ccf-9eb2-1af50eb19cb4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:54.145035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d056e4-730c-4ccf-9eb2-1af50eb19cb4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:54.147285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d056e4-730c-4ccf-9eb2-1af50eb19cb4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a5d056e4-730c-4ccf-9eb2-1af50eb19cb4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:54.159264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:54.160395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:54.161425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 321/659] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:54.162068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893fcf0-89e5-41fe-afed-4727ab86c36c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:54.162068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893fcf0-89e5-41fe-afed-4727ab86c36c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:54.162207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893fcf0-89e5-41fe-afed-4727ab86c36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:54.181024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5893fcf0-89e5-41fe-afed-4727ab86c36c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:54.182261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5893fcf0-89e5-41fe-afed-4727ab86c36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:54.192747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 339/660] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:54.192747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2275d-3586-409f-94d5-d086da24bf8b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:54.257350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 322/661] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:54.257350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893fcf0-89e5-41fe-afed-4727ab86c36c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:54.307440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2275d-3586-409f-94d5-d086da24bf8b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:54.350912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:54.355646 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1b247-a0ca-45cf-a187-9749b9a45035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:54.356302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1b247-a0ca-45cf-a187-9749b9a45035 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:54.356588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1b247-a0ca-45cf-a187-9749b9a45035 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:54.356864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1b247-a0ca-45cf-a187-9749b9a45035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:54.368356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1b247-a0ca-45cf-a187-9749b9a45035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:54.370020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1b247-a0ca-45cf-a187-9749b9a45035 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:54.378660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:54.381012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893fcf0-89e5-41fe-afed-4727ab86c36c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-47e1b247-a0ca-45cf-a187-9749b9a45035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:54.389441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:54.390396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:54.391322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:54.391322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:54.391322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:54.391322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:54.393112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 340/662] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:54.394291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1b247-a0ca-45cf-a187-9749b9a45035 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:54.424733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12f5f-1a66-49bd-bc07-7d514bc3cf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:54.425692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12f5f-1a66-49bd-bc07-7d514bc3cf17 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:54.426061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12f5f-1a66-49bd-bc07-7d514bc3cf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:54.426249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12f5f-1a66-49bd-bc07-7d514bc3cf17 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:54.427054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c12f5f-1a66-49bd-bc07-7d514bc3cf17 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:54.427361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c12f5f-1a66-49bd-bc07-7d514bc3cf17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:54.428917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c12f5f-1a66-49bd-bc07-7d514bc3cf17 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:54.449298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 323/663] 10.222.0.69 () {56 vars in 1139 bytes} [Thu Jul 28 06:09:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:54.450313 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12f5f-1a66-49bd-bc07-7d514bc3cf17 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:54.506127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1b247-a0ca-45cf-a187-9749b9a45035 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:54.543166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0cb19-0199-4c47-987f-ed2537742bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:54.543814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0cb19-0199-4c47-987f-ed2537742bcf None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:54.544101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0cb19-0199-4c47-987f-ed2537742bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:54.544383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0cb19-0199-4c47-987f-ed2537742bcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:54.556005 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0cb19-0199-4c47-987f-ed2537742bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:54.557593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0cb19-0199-4c47-987f-ed2537742bcf None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:54.561560 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12f5f-1a66-49bd-bc07-7d514bc3cf17 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-47c0cb19-0199-4c47-987f-ed2537742bcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:54.571828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:54.572829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:54.574394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:54.574394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:54.574394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:54.574394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:54.574394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 341/664] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:54.574394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0cb19-0199-4c47-987f-ed2537742bcf None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:54.610211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ebb14-7b64-4f3b-9fce-1d023b4df7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:54.610885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ebb14-7b64-4f3b-9fce-1d023b4df7ab None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:54.611173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ebb14-7b64-4f3b-9fce-1d023b4df7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:54.611742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ebb14-7b64-4f3b-9fce-1d023b4df7ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:54.615585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:54.625170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ebb14-7b64-4f3b-9fce-1d023b4df7ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:54.629488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a83ebb14-7b64-4f3b-9fce-1d023b4df7ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:54.632483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:54.633506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:54.634364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 324/665] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:09:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:54.636163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ebb14-7b64-4f3b-9fce-1d023b4df7ab None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:54.686245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0cb19-0199-4c47-987f-ed2537742bcf None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:54.722239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97924f-80f0-4616-8b4f-fddef6735ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:54.722976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97924f-80f0-4616-8b4f-fddef6735ead None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:54.723265 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97924f-80f0-4616-8b4f-fddef6735ead None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:54.723672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97924f-80f0-4616-8b4f-fddef6735ead None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:54.724377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97924f-80f0-4616-8b4f-fddef6735ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:54.724689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97924f-80f0-4616-8b4f-fddef6735ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:54.726212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97924f-80f0-4616-8b4f-fddef6735ead None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:54.741657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ebb14-7b64-4f3b-9fce-1d023b4df7ab None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:54.744229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 342/666] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:54.745124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97924f-80f0-4616-8b4f-fddef6735ead None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:54.779239 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eed3b-32eb-430b-bea6-7b941f63efef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:54.779867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eed3b-32eb-430b-bea6-7b941f63efef None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:54.780150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eed3b-32eb-430b-bea6-7b941f63efef None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:54.780426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eed3b-32eb-430b-bea6-7b941f63efef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:54.856908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:54.863475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6eed3b-32eb-430b-bea6-7b941f63efef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:54.863631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97924f-80f0-4616-8b4f-fddef6735ead None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:54.865247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6eed3b-32eb-430b-bea6-7b941f63efef None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:54.874836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:54.903450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916d2d4-a175-4ec5-a7e3-fc3fc2a0430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:54.904368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916d2d4-a175-4ec5-a7e3-fc3fc2a0430e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:54.904658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916d2d4-a175-4ec5-a7e3-fc3fc2a0430e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:54.904942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916d2d4-a175-4ec5-a7e3-fc3fc2a0430e None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:54.905699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916d2d4-a175-4ec5-a7e3-fc3fc2a0430e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:54.905995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916d2d4-a175-4ec5-a7e3-fc3fc2a0430e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:54.907430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916d2d4-a175-4ec5-a7e3-fc3fc2a0430e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:54.923621 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 343/667] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:54.923621 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916d2d4-a175-4ec5-a7e3-fc3fc2a0430e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:54.985337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 325/668] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:54.985337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eed3b-32eb-430b-bea6-7b941f63efef None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.036976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916d2d4-a175-4ec5-a7e3-fc3fc2a0430e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.072103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86318fb-f00d-4eac-ab83-7cac0b8cf2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.072731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86318fb-f00d-4eac-ab83-7cac0b8cf2cc None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.073019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86318fb-f00d-4eac-ab83-7cac0b8cf2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.073298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86318fb-f00d-4eac-ab83-7cac0b8cf2cc None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.074049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86318fb-f00d-4eac-ab83-7cac0b8cf2cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:55.074509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86318fb-f00d-4eac-ab83-7cac0b8cf2cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.075907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86318fb-f00d-4eac-ab83-7cac0b8cf2cc None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.084080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:55.084555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 344/669] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:09:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:55.085149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86318fb-f00d-4eac-ab83-7cac0b8cf2cc None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.091722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eed3b-32eb-430b-bea6-7b941f63efef None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.122791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:55.128634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ab58b-2b87-4cf6-8a12-0bb5d4254151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.129294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ab58b-2b87-4cf6-8a12-0bb5d4254151 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.129605 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ab58b-2b87-4cf6-8a12-0bb5d4254151 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.129886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ab58b-2b87-4cf6-8a12-0bb5d4254151 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.136849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ab58b-2b87-4cf6-8a12-0bb5d4254151 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.138901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ab58b-2b87-4cf6-8a12-0bb5d4254151 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7e4ab58b-2b87-4cf6-8a12-0bb5d4254151 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:55.150464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:55.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:55.152254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:55.152254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:55.152254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:55.152254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:55.152254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:55.152254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:55.152254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:55.152254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:55.152254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 326/670] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:55.152734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ab58b-2b87-4cf6-8a12-0bb5d4254151 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.206876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86318fb-f00d-4eac-ab83-7cac0b8cf2cc None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.243742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc35cd5-5500-4b21-aecc-9fada6f11d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.244732 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc35cd5-5500-4b21-aecc-9fada6f11d85 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.245220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc35cd5-5500-4b21-aecc-9fada6f11d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.245536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc35cd5-5500-4b21-aecc-9fada6f11d85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.252731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc35cd5-5500-4b21-aecc-9fada6f11d85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.254222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc35cd5-5500-4b21-aecc-9fada6f11d85 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.261518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ab58b-2b87-4cf6-8a12-0bb5d4254151 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-5fc35cd5-5500-4b21-aecc-9fada6f11d85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:55.267025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:55.268477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:55.269417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:55.269417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:55.269417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:55.269417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:55.269417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:55.269417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:55.269417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:55.269417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:55.269417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 345/671] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:55.269920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc35cd5-5500-4b21-aecc-9fada6f11d85 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.301931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bdf30-c84c-4ba0-bc7c-03a5a97264ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.302587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bdf30-c84c-4ba0-bc7c-03a5a97264ad None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.302876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bdf30-c84c-4ba0-bc7c-03a5a97264ad None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.303154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bdf30-c84c-4ba0-bc7c-03a5a97264ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.314529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bdf30-c84c-4ba0-bc7c-03a5a97264ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.316216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bdf30-c84c-4ba0-bc7c-03a5a97264ad None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-594bdf30-c84c-4ba0-bc7c-03a5a97264ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:55.332266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:55.333645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:55.334731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:55.334731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:55.334731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:55.334731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:55.334731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 327/672] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:55.338527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bdf30-c84c-4ba0-bc7c-03a5a97264ad None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.354664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:55.376431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:55.389339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc35cd5-5500-4b21-aecc-9fada6f11d85 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.428939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03758c26-c331-42a2-af5d-8fb0282fdce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.429779 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03758c26-c331-42a2-af5d-8fb0282fdce5 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.430172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03758c26-c331-42a2-af5d-8fb0282fdce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.430563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03758c26-c331-42a2-af5d-8fb0282fdce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.442119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03758c26-c331-42a2-af5d-8fb0282fdce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.443587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03758c26-c331-42a2-af5d-8fb0282fdce5 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-03758c26-c331-42a2-af5d-8fb0282fdce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:55.458310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:55.459269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:55.460156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:55.460156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:55.460156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:55.460156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:55.460156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 346/673] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:55.460156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03758c26-c331-42a2-af5d-8fb0282fdce5 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.466634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bdf30-c84c-4ba0-bc7c-03a5a97264ad None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.506980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e1447-436a-40e6-bc47-bdc73f04262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.507828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e1447-436a-40e6-bc47-bdc73f04262f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.508212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e1447-436a-40e6-bc47-bdc73f04262f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.508590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e1447-436a-40e6-bc47-bdc73f04262f None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.509487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e1447-436a-40e6-bc47-bdc73f04262f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:55.509899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e1447-436a-40e6-bc47-bdc73f04262f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.511452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e1447-436a-40e6-bc47-bdc73f04262f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.530209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 328/674] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:55.530209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e1447-436a-40e6-bc47-bdc73f04262f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.574342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03758c26-c331-42a2-af5d-8fb0282fdce5 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.597778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:55.612213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a2002-90ce-42a2-a351-f1fbcade91db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.613156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a2002-90ce-42a2-a351-f1fbcade91db None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.613156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a2002-90ce-42a2-a351-f1fbcade91db None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.613433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a2002-90ce-42a2-a351-f1fbcade91db None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.614201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a2002-90ce-42a2-a351-f1fbcade91db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:55.614521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a2002-90ce-42a2-a351-f1fbcade91db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.616050 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a2002-90ce-42a2-a351-f1fbcade91db None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.633640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 347/675] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:55.633640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a2002-90ce-42a2-a351-f1fbcade91db None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.633640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:55.639426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e1447-436a-40e6-bc47-bdc73f04262f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.676123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafbf26-3594-49ae-baeb-7d349fab0d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.676754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafbf26-3594-49ae-baeb-7d349fab0d6f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.677039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafbf26-3594-49ae-baeb-7d349fab0d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.677337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafbf26-3594-49ae-baeb-7d349fab0d6f None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.678043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafbf26-3594-49ae-baeb-7d349fab0d6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:55.678341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafbf26-3594-49ae-baeb-7d349fab0d6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.679746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafbf26-3594-49ae-baeb-7d349fab0d6f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.691800 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 329/676] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:55.692621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafbf26-3594-49ae-baeb-7d349fab0d6f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.742474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a2002-90ce-42a2-a351-f1fbcade91db None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.776417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d1d85-5ad8-4b58-b2a4-64c25f1a177c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.777039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d1d85-5ad8-4b58-b2a4-64c25f1a177c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.777353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d1d85-5ad8-4b58-b2a4-64c25f1a177c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.777633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d1d85-5ad8-4b58-b2a4-64c25f1a177c None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.778332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d1d85-5ad8-4b58-b2a4-64c25f1a177c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:55.778641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d1d85-5ad8-4b58-b2a4-64c25f1a177c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.780027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d1d85-5ad8-4b58-b2a4-64c25f1a177c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.791572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 348/677] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:55.792373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d1d85-5ad8-4b58-b2a4-64c25f1a177c None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.799010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafbf26-3594-49ae-baeb-7d349fab0d6f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.824814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:55.833754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f500e7-f45f-49a4-b710-92dea4fdce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.834446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f500e7-f45f-49a4-b710-92dea4fdce02 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.834792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f500e7-f45f-49a4-b710-92dea4fdce02 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.835020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f500e7-f45f-49a4-b710-92dea4fdce02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.839352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f500e7-f45f-49a4-b710-92dea4fdce02 None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=cc4fe04b11ec412e8f4eb67c6e54b8f7) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:55.839352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f500e7-f45f-49a4-b710-92dea4fdce02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.839352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f500e7-f45f-49a4-b710-92dea4fdce02 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.843511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 330/678] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/role_assignments?user.id=cc4fe04b11ec412e8f4eb67c6e54b8f7&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:55.844309 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f500e7-f45f-49a4-b710-92dea4fdce02 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.859793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:55.906638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d1d85-5ad8-4b58-b2a4-64c25f1a177c None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:55.954115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fbc00-5655-4849-ae26-2acc3d8573cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:55.954881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fbc00-5655-4849-ae26-2acc3d8573cd None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.955699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fbc00-5655-4849-ae26-2acc3d8573cd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.956071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fbc00-5655-4849-ae26-2acc3d8573cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.956912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fbc00-5655-4849-ae26-2acc3d8573cd None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=cbbbf34c238d44db8588d56f35827390) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:55.957229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fbc00-5655-4849-ae26-2acc3d8573cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:55.958758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fbc00-5655-4849-ae26-2acc3d8573cd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:55.973652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 349/679] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/role_assignments?user.id=cbbbf34c238d44db8588d56f35827390&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:55.973652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d6b24-ffbc-42fa-836e-d4b109f510e5 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:55.973652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d6b24-ffbc-42fa-836e-d4b109f510e5 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:55.973652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d6b24-ffbc-42fa-836e-d4b109f510e5 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:55.973652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 350/680] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:55.977397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d6b24-ffbc-42fa-836e-d4b109f510e5 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:55.977625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f500e7-f45f-49a4-b710-92dea4fdce02 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:56.020575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552d05c-06d2-43cc-8df5-5ae8862e93db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:56.021844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552d05c-06d2-43cc-8df5-5ae8862e93db None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:56.022133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552d05c-06d2-43cc-8df5-5ae8862e93db None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:56.022707 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552d05c-06d2-43cc-8df5-5ae8862e93db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:56.032094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552d05c-06d2-43cc-8df5-5ae8862e93db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:56.036025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552d05c-06d2-43cc-8df5-5ae8862e93db None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-5552d05c-06d2-43cc-8df5-5ae8862e93db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:56.053041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:56.054027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:56.054893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:56.054893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:56.054893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:56.054893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:56.054893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:56.054893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:56.054893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:56.054893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:56.055598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 331/681] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:56.060093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e518d-f139-4dde-a958-43598cf26c82 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:56.060093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e518d-f139-4dde-a958-43598cf26c82 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:56.060093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e518d-f139-4dde-a958-43598cf26c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:56.083392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1f6e518d-f139-4dde-a958-43598cf26c82 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:56.084838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1f6e518d-f139-4dde-a958-43598cf26c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:56.130402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d6b24-ffbc-42fa-836e-d4b109f510e5 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:56.163537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:56.167581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:56.176796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8015f-6918-4120-b5e0-224b395c665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:56.177488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8015f-6918-4120-b5e0-224b395c665e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:56.181451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8015f-6918-4120-b5e0-224b395c665e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:56.181451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8015f-6918-4120-b5e0-224b395c665e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:56.188634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8015f-6918-4120-b5e0-224b395c665e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:56.190937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8015f-6918-4120-b5e0-224b395c665e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:56.192564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 332/682] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:56.194505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e518d-f139-4dde-a958-43598cf26c82 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-7ea8015f-6918-4120-b5e0-224b395c665e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:56.204684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:56.206660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:56.208116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 351/683] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:09:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:56.212451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8015f-6918-4120-b5e0-224b395c665e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:56.337882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e518d-f139-4dde-a958-43598cf26c82 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:56.342897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8015f-6918-4120-b5e0-224b395c665e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:56.394208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced21df4-45d8-4d0c-9a11-ca33c8ad363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:56.398875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced21df4-45d8-4d0c-9a11-ca33c8ad363d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:56.399431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced21df4-45d8-4d0c-9a11-ca33c8ad363d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:56.400115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced21df4-45d8-4d0c-9a11-ca33c8ad363d None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:56.401054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa78a7b-c18f-4f91-a50b-c73b6e4e9799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:56.401885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa78a7b-c18f-4f91-a50b-c73b6e4e9799 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:56.402279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa78a7b-c18f-4f91-a50b-c73b6e4e9799 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:56.402634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced21df4-45d8-4d0c-9a11-ca33c8ad363d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:56.402813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa78a7b-c18f-4f91-a50b-c73b6e4e9799 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:56.403575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced21df4-45d8-4d0c-9a11-ca33c8ad363d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:56.403753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa78a7b-c18f-4f91-a50b-c73b6e4e9799 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:56.404142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa78a7b-c18f-4f91-a50b-c73b6e4e9799 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:56.405750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa78a7b-c18f-4f91-a50b-c73b6e4e9799 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:56.407329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced21df4-45d8-4d0c-9a11-ca33c8ad363d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:56.430812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 333/684] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:56.431918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced21df4-45d8-4d0c-9a11-ca33c8ad363d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:56.451148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 352/685] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 28 06:09:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:56.451148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa78a7b-c18f-4f91-a50b-c73b6e4e9799 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:56.497552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:56.513120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:56.589792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced21df4-45d8-4d0c-9a11-ca33c8ad363d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:56.590912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa78a7b-c18f-4f91-a50b-c73b6e4e9799 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:56.644440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275f382-5487-4180-8458-c1f0b9ffad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:56.645188 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275f382-5487-4180-8458-c1f0b9ffad81 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:56.645831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275f382-5487-4180-8458-c1f0b9ffad81 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:56.646287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275f382-5487-4180-8458-c1f0b9ffad81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:56.648510 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275f382-5487-4180-8458-c1f0b9ffad81 None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=05574eace38f4b6d945c7b1927fdadd8) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:56.648510 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275f382-5487-4180-8458-c1f0b9ffad81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:56.649083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275f382-5487-4180-8458-c1f0b9ffad81 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:56.657739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406177c-5706-44a3-8ef7-db9eebb60504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:56.657942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 353/686] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:56 2022] GET /identity/v3/role_assignments?user.id=05574eace38f4b6d945c7b1927fdadd8&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:56.659598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275f382-5487-4180-8458-c1f0b9ffad81 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:56.660437 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406177c-5706-44a3-8ef7-db9eebb60504 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:56.661094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406177c-5706-44a3-8ef7-db9eebb60504 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:56.661699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406177c-5706-44a3-8ef7-db9eebb60504 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:56.678879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406177c-5706-44a3-8ef7-db9eebb60504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c406177c-5706-44a3-8ef7-db9eebb60504 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:09:56.688326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:56.689486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:09:56.690346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:56.693391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 334/687] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:09:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:56.699861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406177c-5706-44a3-8ef7-db9eebb60504 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:56.811899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275f382-5487-4180-8458-c1f0b9ffad81 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:56.836915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:56.896519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406177c-5706-44a3-8ef7-db9eebb60504 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:56.903474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4bbcb-f43a-4b0c-af52-8f83b4bd7393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:56.904164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:56.904485 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4bbcb-f43a-4b0c-af52-8f83b4bd7393 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:56.904938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4bbcb-f43a-4b0c-af52-8f83b4bd7393 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:56.905405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4bbcb-f43a-4b0c-af52-8f83b4bd7393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:56.966770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b3b54-8ae4-4501-9368-b622a5a3fbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:56.967819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b3b54-8ae4-4501-9368-b622a5a3fbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:56.968306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b3b54-8ae4-4501-9368-b622a5a3fbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:56.968848 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b3b54-8ae4-4501-9368-b622a5a3fbe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:56.970422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b3b54-8ae4-4501-9368-b622a5a3fbe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:56.971769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b3b54-8ae4-4501-9368-b622a5a3fbe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:56.974243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b3b54-8ae4-4501-9368-b622a5a3fbe4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:56.989527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 335/688] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:09:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:56.995764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b3b54-8ae4-4501-9368-b622a5a3fbe4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:57.016264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4bbcb-f43a-4b0c-af52-8f83b4bd7393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:57.018596 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4bbcb-f43a-4b0c-af52-8f83b4bd7393 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:57.148620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b3b54-8ae4-4501-9368-b622a5a3fbe4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:57.151599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 354/689] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:57.157796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4bbcb-f43a-4b0c-af52-8f83b4bd7393 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:57.172607 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:57.212981 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a48deb-35c2-46b3-af88-bbed58ddfb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:57.214364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a48deb-35c2-46b3-af88-bbed58ddfb4e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:57.214653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a48deb-35c2-46b3-af88-bbed58ddfb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:57.215314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a48deb-35c2-46b3-af88-bbed58ddfb4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:57.234650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a48deb-35c2-46b3-af88-bbed58ddfb4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:57.239966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a48deb-35c2-46b3-af88-bbed58ddfb4e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e2a48deb-35c2-46b3-af88-bbed58ddfb4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:57.272832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:57.274009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:57.275027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:57.275027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:57.275027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:57.275027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:57.282499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 336/690] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:57.292738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a48deb-35c2-46b3-af88-bbed58ddfb4e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:57.328602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:57.339532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4bbcb-f43a-4b0c-af52-8f83b4bd7393 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:57.382818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48417670-198b-4140-a595-1eb9f787386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:57.383939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48417670-198b-4140-a595-1eb9f787386b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:57.384434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48417670-198b-4140-a595-1eb9f787386b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:57.384952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48417670-198b-4140-a595-1eb9f787386b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:57.397273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48417670-198b-4140-a595-1eb9f787386b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:57.399386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48417670-198b-4140-a595-1eb9f787386b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-48417670-198b-4140-a595-1eb9f787386b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:57.420506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:57.421528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:57.422442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:57.422442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:57.422442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:57.422442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:57.425890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 355/691] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:57.431873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48417670-198b-4140-a595-1eb9f787386b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:57.485870 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a48deb-35c2-46b3-af88-bbed58ddfb4e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:57.512377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:57.562666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48417670-198b-4140-a595-1eb9f787386b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:57.588693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d6a66-ff47-4b69-a0f4-144b0b4f2952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:57.589345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d6a66-ff47-4b69-a0f4-144b0b4f2952 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:57.589638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d6a66-ff47-4b69-a0f4-144b0b4f2952 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:57.589915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d6a66-ff47-4b69-a0f4-144b0b4f2952 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:57.590728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d6a66-ff47-4b69-a0f4-144b0b4f2952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:57.591043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d6a66-ff47-4b69-a0f4-144b0b4f2952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:57.592544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d6a66-ff47-4b69-a0f4-144b0b4f2952 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:57.604530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd308c5-6ab2-48e3-943f-234d64f3f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:57.606002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd308c5-6ab2-48e3-943f-234d64f3f792 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:57.606522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd308c5-6ab2-48e3-943f-234d64f3f792 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:57.607007 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd308c5-6ab2-48e3-943f-234d64f3f792 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:57.607924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd308c5-6ab2-48e3-943f-234d64f3f792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:57.608486 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd308c5-6ab2-48e3-943f-234d64f3f792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:57.610112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd308c5-6ab2-48e3-943f-234d64f3f792 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:57.615879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 337/692] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:57.623982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d6a66-ff47-4b69-a0f4-144b0b4f2952 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:57.639657 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 356/693] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:57.645228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd308c5-6ab2-48e3-943f-234d64f3f792 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:57.716920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:57.743247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d6a66-ff47-4b69-a0f4-144b0b4f2952 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:57.785895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce809a-b9cb-4c23-81a9-a7a15678c86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:57.786899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce809a-b9cb-4c23-81a9-a7a15678c86f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:57.787389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce809a-b9cb-4c23-81a9-a7a15678c86f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:57.787869 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce809a-b9cb-4c23-81a9-a7a15678c86f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:57.796337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce809a-b9cb-4c23-81a9-a7a15678c86f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:57.798081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce809a-b9cb-4c23-81a9-a7a15678c86f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c2ce809a-b9cb-4c23-81a9-a7a15678c86f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:09:57.813039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:57.814097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:57.815852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:57.815852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:57.815852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:09:57.815852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:09:57.815852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:09:57.815852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:09:57.815852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:09:57.815852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:09:57.815852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:09:57.815852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:57.819120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 338/694] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 28 06:09:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:57.825054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce809a-b9cb-4c23-81a9-a7a15678c86f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:57.946763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce809a-b9cb-4c23-81a9-a7a15678c86f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:57.951533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd308c5-6ab2-48e3-943f-234d64f3f792 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:57.989107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f1161-fd7d-471a-aeee-589e19f6c61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:57.990105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1161-fd7d-471a-aeee-589e19f6c61b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:57.990617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1161-fd7d-471a-aeee-589e19f6c61b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:57.991109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1161-fd7d-471a-aeee-589e19f6c61b None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:57.992184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f1161-fd7d-471a-aeee-589e19f6c61b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:57.992715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f1161-fd7d-471a-aeee-589e19f6c61b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:57.995085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f1161-fd7d-471a-aeee-589e19f6c61b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:57.998540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:58.006019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:58.019712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 339/695] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 28 06:09:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:58.027757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f1161-fd7d-471a-aeee-589e19f6c61b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:58.040543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485edc51-ad6a-4332-95e3-50ef3914ccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:58.041217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485edc51-ad6a-4332-95e3-50ef3914ccb7 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:58.041531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485edc51-ad6a-4332-95e3-50ef3914ccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:58.041869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485edc51-ad6a-4332-95e3-50ef3914ccb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:58.051842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485edc51-ad6a-4332-95e3-50ef3914ccb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:58.053358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485edc51-ad6a-4332-95e3-50ef3914ccb7 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-485edc51-ad6a-4332-95e3-50ef3914ccb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:58.075884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:58.076853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:58.077786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:58.077786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:58.077786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:58.077786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:58.077786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:58.077786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:58.077786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:58.077786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:58.078663 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 357/696] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:09:58.090355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485edc51-ad6a-4332-95e3-50ef3914ccb7 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:58.169672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f1161-fd7d-471a-aeee-589e19f6c61b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:58.208215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f17150-f04e-420a-872a-9753821c810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:58.209250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f17150-f04e-420a-872a-9753821c810c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:58.209773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f17150-f04e-420a-872a-9753821c810c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:58.210332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f17150-f04e-420a-872a-9753821c810c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:58.289705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:58.294336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f17150-f04e-420a-872a-9753821c810c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:58.296371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f17150-f04e-420a-872a-9753821c810c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:58.301346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485edc51-ad6a-4332-95e3-50ef3914ccb7 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:58.379791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22f81c-02ee-4399-842e-fd70f3f1aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:58.381454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22f81c-02ee-4399-842e-fd70f3f1aab0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:58.381957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22f81c-02ee-4399-842e-fd70f3f1aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:58.382445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22f81c-02ee-4399-842e-fd70f3f1aab0 None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:58.383440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22f81c-02ee-4399-842e-fd70f3f1aab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:58.384110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22f81c-02ee-4399-842e-fd70f3f1aab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:58.385950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22f81c-02ee-4399-842e-fd70f3f1aab0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:58.401808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 358/697] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:58.403218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b8f48-a485-4c71-b572-b6af658406e3 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:58.403524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b8f48-a485-4c71-b572-b6af658406e3 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:58.403818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b8f48-a485-4c71-b572-b6af658406e3 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:58.405521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 359/698] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:58.407735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b8f48-a485-4c71-b572-b6af658406e3 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:58.453581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 340/699] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:09:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:09:58.455036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43235eb-fed7-4d41-8e28-f3a2ec3137bb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:58.455341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43235eb-fed7-4d41-8e28-f3a2ec3137bb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:58.455605 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43235eb-fed7-4d41-8e28-f3a2ec3137bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:58.466271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:58.477256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f43235eb-fed7-4d41-8e28-f3a2ec3137bb None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:58.478514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f43235eb-fed7-4d41-8e28-f3a2ec3137bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:58.533607 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b8f48-a485-4c71-b572-b6af658406e3 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:58.554310 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 341/700] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:58.555532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43235eb-fed7-4d41-8e28-f3a2ec3137bb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:58.577687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5bb92-9848-4fa7-935b-8623b22b97dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:58.578495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5bb92-9848-4fa7-935b-8623b22b97dc None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:58.578852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5bb92-9848-4fa7-935b-8623b22b97dc None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:58.579204 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5bb92-9848-4fa7-935b-8623b22b97dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:58.580837 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5bb92-9848-4fa7-935b-8623b22b97dc None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=77571de30e934d6590ff7b58a000f5e2) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:58.581277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5bb92-9848-4fa7-935b-8623b22b97dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:58.582973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5bb92-9848-4fa7-935b-8623b22b97dc None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:58.590442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:58.601342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 360/701] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:58 2022] GET /identity/v3/role_assignments?user.id=77571de30e934d6590ff7b58a000f5e2&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:58.601342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd5ada-6d3b-40de-a954-fe9e0fd59238 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:58.601342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd5ada-6d3b-40de-a954-fe9e0fd59238 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:58.601342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd5ada-6d3b-40de-a954-fe9e0fd59238 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:58.601342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 361/702] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:58.605826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd5ada-6d3b-40de-a954-fe9e0fd59238 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:58.684595 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43235eb-fed7-4d41-8e28-f3a2ec3137bb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:58.724003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfe48a-427f-4691-8a9d-e13caf157d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:58.724672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe48a-427f-4691-8a9d-e13caf157d37 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:58.724960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe48a-427f-4691-8a9d-e13caf157d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:58.725239 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe48a-427f-4691-8a9d-e13caf157d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:58.734193 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd5ada-6d3b-40de-a954-fe9e0fd59238 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:58.737505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfe48a-427f-4691-8a9d-e13caf157d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:58.737846 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:58.739453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfe48a-427f-4691-8a9d-e13caf157d37 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-26bfe48a-427f-4691-8a9d-e13caf157d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:58.756851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:58.757877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:58.758936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:58.758936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:58.758936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:58.758936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:58.762808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 342/703] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:58.764254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae285b-1bc8-4022-b2b8-626cfa23dcac None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:58.764544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae285b-1bc8-4022-b2b8-626cfa23dcac None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:58.765566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae285b-1bc8-4022-b2b8-626cfa23dcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:58.783766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-80ae285b-1bc8-4022-b2b8-626cfa23dcac None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:58.784254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e093cb-12e3-4a34-b111-7cebb4752e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:58.784947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e093cb-12e3-4a34-b111-7cebb4752e9a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:58.785071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-80ae285b-1bc8-4022-b2b8-626cfa23dcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:58.785278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e093cb-12e3-4a34-b111-7cebb4752e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:58.785548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e093cb-12e3-4a34-b111-7cebb4752e9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:58.798392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e093cb-12e3-4a34-b111-7cebb4752e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:58.801169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e093cb-12e3-4a34-b111-7cebb4752e9a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-75e093cb-12e3-4a34-b111-7cebb4752e9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:09:58.818231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:58.819325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:58.820346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:09:58.820346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:09:58.820346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:09:58.820346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:58.825031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 362/704] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:09:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:58.826014 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e093cb-12e3-4a34-b111-7cebb4752e9a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:58.875377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 343/705] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:58.876345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae285b-1bc8-4022-b2b8-626cfa23dcac None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:58.886572 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:58.964933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e093cb-12e3-4a34-b111-7cebb4752e9a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:59.001812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7c239-4128-4538-bca6-2aee86e95919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:59.002445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7c239-4128-4538-bca6-2aee86e95919 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:59.002737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7c239-4128-4538-bca6-2aee86e95919 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:59.003019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7c239-4128-4538-bca6-2aee86e95919 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:59.003724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7c239-4128-4538-bca6-2aee86e95919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:59.004038 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7c239-4128-4538-bca6-2aee86e95919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:59.005650 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7c239-4128-4538-bca6-2aee86e95919 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:59.015723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae285b-1bc8-4022-b2b8-626cfa23dcac None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:59.020237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 363/706] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:59.021278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7c239-4128-4538-bca6-2aee86e95919 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:59.046423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:59.064907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c053ae-dbce-447b-9776-a1eb98a8fcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:59.065585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c053ae-dbce-447b-9776-a1eb98a8fcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:59.065871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c053ae-dbce-447b-9776-a1eb98a8fcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:59.066149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c053ae-dbce-447b-9776-a1eb98a8fcd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:59.066859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c053ae-dbce-447b-9776-a1eb98a8fcd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:59.067162 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c053ae-dbce-447b-9776-a1eb98a8fcd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:59.068607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c053ae-dbce-447b-9776-a1eb98a8fcd6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:59.085196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 344/707] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:59.086173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c053ae-dbce-447b-9776-a1eb98a8fcd6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:59.144445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7c239-4128-4538-bca6-2aee86e95919 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:59.184000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df505cb-7274-4e5b-8d3e-78ec10181833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:59.184767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df505cb-7274-4e5b-8d3e-78ec10181833 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:59.185130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df505cb-7274-4e5b-8d3e-78ec10181833 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:59.185514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df505cb-7274-4e5b-8d3e-78ec10181833 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:59.186285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df505cb-7274-4e5b-8d3e-78ec10181833 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:59.187696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df505cb-7274-4e5b-8d3e-78ec10181833 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:59.197746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-0df505cb-7274-4e5b-8d3e-78ec10181833 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:59.200022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:59.200992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:09:59.201894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:09:59.206830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 364/708] 10.222.0.69 () {56 vars in 1129 bytes} [Thu Jul 28 06:09:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:59.207807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df505cb-7274-4e5b-8d3e-78ec10181833 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:59.218722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c053ae-dbce-447b-9776-a1eb98a8fcd6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:59.265078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8268c3-4351-489f-88b1-44fae319e698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:59.266070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8268c3-4351-489f-88b1-44fae319e698 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:59.266606 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8268c3-4351-489f-88b1-44fae319e698 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:59.267061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8268c3-4351-489f-88b1-44fae319e698 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:59.276630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8268c3-4351-489f-88b1-44fae319e698 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:59.278558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8268c3-4351-489f-88b1-44fae319e698 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-9a8268c3-4351-489f-88b1-44fae319e698 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:09:59.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:09:59.295299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:09:59.296404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:09:59.296404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:09:59.296404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:09:59.296404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:09:59.296404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:09:59.296404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:09:59.296404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:09:59.296404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:09:59.298577 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 345/709] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:09:59.299816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8268c3-4351-489f-88b1-44fae319e698 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:59.312591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:59.340000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df505cb-7274-4e5b-8d3e-78ec10181833 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:59.396078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc011e7f-e7ca-4779-a75d-ad573e4c8312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:59.398098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc011e7f-e7ca-4779-a75d-ad573e4c8312 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:59.398868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc011e7f-e7ca-4779-a75d-ad573e4c8312 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:59.399364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc011e7f-e7ca-4779-a75d-ad573e4c8312 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:59.400571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc011e7f-e7ca-4779-a75d-ad573e4c8312 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:59.402903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc011e7f-e7ca-4779-a75d-ad573e4c8312 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:59.505119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 365/710] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:09:59 2022] POST /identity/v3/users => generated 254 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:09:59.506238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc011e7f-e7ca-4779-a75d-ad573e4c8312 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:59.564096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8268c3-4351-489f-88b1-44fae319e698 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:59.597762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:59.607935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97893f1c-534e-4835-8b66-2f5d5147f15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:59.608939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97893f1c-534e-4835-8b66-2f5d5147f15c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:59.609530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97893f1c-534e-4835-8b66-2f5d5147f15c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:59.613397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97893f1c-534e-4835-8b66-2f5d5147f15c None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:59.613397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97893f1c-534e-4835-8b66-2f5d5147f15c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:59.613397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97893f1c-534e-4835-8b66-2f5d5147f15c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:59.613397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97893f1c-534e-4835-8b66-2f5d5147f15c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:59.619094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 346/711] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 28 06:09:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:59.620796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b078c-ac9d-457d-9683-6ec8ef640f1a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:59.621437 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b078c-ac9d-457d-9683-6ec8ef640f1a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:59.621944 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b078c-ac9d-457d-9683-6ec8ef640f1a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:59.623698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 347/712] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:09:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:09:59.624983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b078c-ac9d-457d-9683-6ec8ef640f1a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:59.639680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc011e7f-e7ca-4779-a75d-ad573e4c8312 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:59.689284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba385ecb-fdb5-425f-81a2-590b121587a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:59.689957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba385ecb-fdb5-425f-81a2-590b121587a2 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:59.691573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba385ecb-fdb5-425f-81a2-590b121587a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:59.691906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba385ecb-fdb5-425f-81a2-590b121587a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:59.692819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba385ecb-fdb5-425f-81a2-590b121587a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:59.693259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba385ecb-fdb5-425f-81a2-590b121587a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:59.694750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba385ecb-fdb5-425f-81a2-590b121587a2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:59.700034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:59.716367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 366/713] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:09:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:59.716367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f04b-e1f8-4d9d-a63e-4db4d878b886 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:59.716367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f04b-e1f8-4d9d-a63e-4db4d878b886 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:59.716367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f04b-e1f8-4d9d-a63e-4db4d878b886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:59.736772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-0d13f04b-e1f8-4d9d-a63e-4db4d878b886 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:09:59.738403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0d13f04b-e1f8-4d9d-a63e-4db4d878b886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:09:59.779336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b078c-ac9d-457d-9683-6ec8ef640f1a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:09:59.824299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 367/714] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:09:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:09:59.827858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13f04b-e1f8-4d9d-a63e-4db4d878b886 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:59.839316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74578d84-35a4-456d-b43d-51059cfac7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:09:59.839957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74578d84-35a4-456d-b43d-51059cfac7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:09:59.840246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74578d84-35a4-456d-b43d-51059cfac7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:09:59.840527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74578d84-35a4-456d-b43d-51059cfac7e1 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:09:59.841227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74578d84-35a4-456d-b43d-51059cfac7e1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:09:59.841641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74578d84-35a4-456d-b43d-51059cfac7e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:09:59.843027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74578d84-35a4-456d-b43d-51059cfac7e1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:09:59.853518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 348/715] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 28 06:09:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:09:59.854842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74578d84-35a4-456d-b43d-51059cfac7e1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:09:59.924490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:09:59.987021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74578d84-35a4-456d-b43d-51059cfac7e1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:00.039971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa79a6-3f48-410f-8600-8da62ce0b06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:00.040653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa79a6-3f48-410f-8600-8da62ce0b06a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:00.040946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa79a6-3f48-410f-8600-8da62ce0b06a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:00.041232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa79a6-3f48-410f-8600-8da62ce0b06a None admin] PATH_INFO: `/v3/services/network` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:00.041920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa79a6-3f48-410f-8600-8da62ce0b06a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:00.043318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa79a6-3f48-410f-8600-8da62ce0b06a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:00.055453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13f04b-e1f8-4d9d-a63e-4db4d878b886 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6faa79a6-3f48-410f-8600-8da62ce0b06a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:00.063444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:00.064568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:00.067382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 349/716] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:09:59 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:00.074333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa79a6-3f48-410f-8600-8da62ce0b06a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:00.097639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:00.108977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e44b3-e7b0-4a88-98e8-610d58e8623b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:00.109569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e44b3-e7b0-4a88-98e8-610d58e8623b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:00.109856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e44b3-e7b0-4a88-98e8-610d58e8623b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:00.110280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e44b3-e7b0-4a88-98e8-610d58e8623b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:00.111836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e44b3-e7b0-4a88-98e8-610d58e8623b None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=5706aa2d6bf54f05badf9588f51f4677) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:00.112387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e44b3-e7b0-4a88-98e8-610d58e8623b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:00.116136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e44b3-e7b0-4a88-98e8-610d58e8623b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:00.124054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 368/717] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:09:59 2022] GET /identity/v3/role_assignments?user.id=5706aa2d6bf54f05badf9588f51f4677&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:00.236179 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa79a6-3f48-410f-8600-8da62ce0b06a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:00.262528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:00.283838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287cff3-3bf9-4025-84d0-49935498fffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:00.284621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287cff3-3bf9-4025-84d0-49935498fffa None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:00.285420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287cff3-3bf9-4025-84d0-49935498fffa None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:00.285420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287cff3-3bf9-4025-84d0-49935498fffa None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:00.285976 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287cff3-3bf9-4025-84d0-49935498fffa None admin] RBAC: Adding query filter params (name=network) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:00.286288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287cff3-3bf9-4025-84d0-49935498fffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:00.287717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287cff3-3bf9-4025-84d0-49935498fffa None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:00.293287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 350/718] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:00.299856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e44b3-e7b0-4a88-98e8-610d58e8623b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:00.412093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e44b3-e7b0-4a88-98e8-610d58e8623b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:00.451905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0fa78-5341-4dd1-b525-8a03f7f43f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:00.452537 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0fa78-5341-4dd1-b525-8a03f7f43f7f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:00.452873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0fa78-5341-4dd1-b525-8a03f7f43f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:00.453161 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0fa78-5341-4dd1-b525-8a03f7f43f7f None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:00.453911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0fa78-5341-4dd1-b525-8a03f7f43f7f None admin] RBAC: Adding query filter params (type=network) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:00.454222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0fa78-5341-4dd1-b525-8a03f7f43f7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:00.456264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0fa78-5341-4dd1-b525-8a03f7f43f7f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:00.462786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 369/719] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:01.055680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a570e-9aae-4672-a5e0-3bcea56ce43a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:01.056109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a570e-9aae-4672-a5e0-3bcea56ce43a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:01.056288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a570e-9aae-4672-a5e0-3bcea56ce43a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:01.057954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 351/720] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:01.063113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030d8d4-1db8-414a-a1b1-63b6b907b1c3 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:01.063411 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030d8d4-1db8-414a-a1b1-63b6b907b1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:01.063694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030d8d4-1db8-414a-a1b1-63b6b907b1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:01.088393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-5030d8d4-1db8-414a-a1b1-63b6b907b1c3 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:01.089636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-5030d8d4-1db8-414a-a1b1-63b6b907b1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:01.159793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:01.180901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 370/721] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:10:01.223550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a570e-9aae-4672-a5e0-3bcea56ce43a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:01.354175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a570e-9aae-4672-a5e0-3bcea56ce43a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:01.414260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7709bf7-dc3d-440e-a5c2-1ccc42ed69b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:01.414919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7709bf7-dc3d-440e-a5c2-1ccc42ed69b3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:01.415205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7709bf7-dc3d-440e-a5c2-1ccc42ed69b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:01.415487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7709bf7-dc3d-440e-a5c2-1ccc42ed69b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:01.426768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7709bf7-dc3d-440e-a5c2-1ccc42ed69b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a7709bf7-dc3d-440e-a5c2-1ccc42ed69b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:01.431715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:01.432680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:01.433556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 352/722] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:01.441593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030d8d4-1db8-414a-a1b1-63b6b907b1c3 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:01.578654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030d8d4-1db8-414a-a1b1-63b6b907b1c3 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:01.618468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc459912-712d-4681-90c5-4f4e2d8aa13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:01.619446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc459912-712d-4681-90c5-4f4e2d8aa13d None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:01.619933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc459912-712d-4681-90c5-4f4e2d8aa13d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:01.620441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc459912-712d-4681-90c5-4f4e2d8aa13d None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:01.621397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc459912-712d-4681-90c5-4f4e2d8aa13d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:01.621935 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc459912-712d-4681-90c5-4f4e2d8aa13d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:01.623500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc459912-712d-4681-90c5-4f4e2d8aa13d None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:01.633807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 371/723] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:10:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:01.639894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7709bf7-dc3d-440e-a5c2-1ccc42ed69b3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:01.698820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:01.771894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7709bf7-dc3d-440e-a5c2-1ccc42ed69b3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:01.812258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849db8b4-b699-47b9-b3ec-50dfeb8c7d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:01.812908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849db8b4-b699-47b9-b3ec-50dfeb8c7d9b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:01.813195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849db8b4-b699-47b9-b3ec-50dfeb8c7d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:01.813497 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849db8b4-b699-47b9-b3ec-50dfeb8c7d9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:01.825218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849db8b4-b699-47b9-b3ec-50dfeb8c7d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:01.826723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849db8b4-b699-47b9-b3ec-50dfeb8c7d9b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-849db8b4-b699-47b9-b3ec-50dfeb8c7d9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:01.841393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:01.842314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:01.843677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:01.843677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:01.843677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:01.843677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:01.843677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 353/724] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:01.848981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc459912-712d-4681-90c5-4f4e2d8aa13d None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:01.903073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:01.959755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc459912-712d-4681-90c5-4f4e2d8aa13d None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:01.994824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d491f-69da-4ab4-ac7a-f9c7ae64b9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:01.995495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d491f-69da-4ab4-ac7a-f9c7ae64b9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:01.995783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d491f-69da-4ab4-ac7a-f9c7ae64b9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:01.996063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d491f-69da-4ab4-ac7a-f9c7ae64b9c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:01.996777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d491f-69da-4ab4-ac7a-f9c7ae64b9c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:01.997110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d491f-69da-4ab4-ac7a-f9c7ae64b9c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:01.998524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d491f-69da-4ab4-ac7a-f9c7ae64b9c9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:02.011098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 372/725] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:02.016797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849db8b4-b699-47b9-b3ec-50dfeb8c7d9b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:02.021259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09accb-ba96-4167-a404-f2c24e3193a2 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.021591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09accb-ba96-4167-a404-f2c24e3193a2 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.021879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09accb-ba96-4167-a404-f2c24e3193a2 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.023451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 373/726] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:02.028387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860131-1f9c-466d-811e-e071a76756bb None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.028669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860131-1f9c-466d-811e-e071a76756bb None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.028955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860131-1f9c-466d-811e-e071a76756bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.063059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-80860131-1f9c-466d-811e-e071a76756bb None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:02.064914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-80860131-1f9c-466d-811e-e071a76756bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:02.132313 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:02.139908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849db8b4-b699-47b9-b3ec-50dfeb8c7d9b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:02.187521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29868a9a-4e34-4c4a-bc8c-4447542caff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:02.188094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29868a9a-4e34-4c4a-bc8c-4447542caff3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.188379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29868a9a-4e34-4c4a-bc8c-4447542caff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.188656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29868a9a-4e34-4c4a-bc8c-4447542caff3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.198332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29868a9a-4e34-4c4a-bc8c-4447542caff3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:02.199974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29868a9a-4e34-4c4a-bc8c-4447542caff3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:02.208484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 374/727] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-29868a9a-4e34-4c4a-bc8c-4447542caff3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:02.213996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:02.215233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:02.216414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:02.216414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:02.216414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:02.216414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:02.216414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:02.216414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:02.216414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:02.216414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:02.216414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:10:02.216414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:02.218364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 354/728] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:02.224169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80860131-1f9c-466d-811e-e071a76756bb None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:02.255072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29868a9a-4e34-4c4a-bc8c-4447542caff3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:02.275107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:02.418733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80860131-1f9c-466d-811e-e071a76756bb None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:02.455872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29868a9a-4e34-4c4a-bc8c-4447542caff3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:02.477121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897a66e-7214-45d2-9c60-62456da73800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:02.477781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897a66e-7214-45d2-9c60-62456da73800 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.478070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897a66e-7214-45d2-9c60-62456da73800 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.478349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897a66e-7214-45d2-9c60-62456da73800 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.479166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897a66e-7214-45d2-9c60-62456da73800 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:02.481910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897a66e-7214-45d2-9c60-62456da73800 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:02.481910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897a66e-7214-45d2-9c60-62456da73800 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:02.505510 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 375/729] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 28 06:10:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:02.506166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9794c-e913-45a2-8d06-f4878b9667d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:02.506844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9794c-e913-45a2-8d06-f4878b9667d4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.507127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9794c-e913-45a2-8d06-f4878b9667d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.507406 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9794c-e913-45a2-8d06-f4878b9667d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.514535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897a66e-7214-45d2-9c60-62456da73800 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:02.524387 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9794c-e913-45a2-8d06-f4878b9667d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-edc9794c-e913-45a2-8d06-f4878b9667d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:02.529714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:02.530652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:10:02.531503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:02.536265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 355/730] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:10:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:02.537541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9794c-e913-45a2-8d06-f4878b9667d4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:02.574116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:02.694854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:02.703048 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897a66e-7214-45d2-9c60-62456da73800 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:02.751166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9794c-e913-45a2-8d06-f4878b9667d4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:02.781033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e07f1-563d-4e08-a14e-1f4f105081cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:02.782110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e07f1-563d-4e08-a14e-1f4f105081cb None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.782653 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e07f1-563d-4e08-a14e-1f4f105081cb None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.783219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e07f1-563d-4e08-a14e-1f4f105081cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.793454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a232f-d3db-4c5d-b9cf-ef55d066bf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:02.793642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a232f-d3db-4c5d-b9cf-ef55d066bf86 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.793938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a232f-d3db-4c5d-b9cf-ef55d066bf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.794221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a232f-d3db-4c5d-b9cf-ef55d066bf86 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.794948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a232f-d3db-4c5d-b9cf-ef55d066bf86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:02.795256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a232f-d3db-4c5d-b9cf-ef55d066bf86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:02.796650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a232f-d3db-4c5d-b9cf-ef55d066bf86 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:02.806187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 356/731] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:10:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:02.818592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a232f-d3db-4c5d-b9cf-ef55d066bf86 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:02.872887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e07f1-563d-4e08-a14e-1f4f105081cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:02.874989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e07f1-563d-4e08-a14e-1f4f105081cb None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:02.927776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:02.950799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a232f-d3db-4c5d-b9cf-ef55d066bf86 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:02.978019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:02.984980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 376/732] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:10:02.986305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a34a0-703a-48a6-8f4b-df878a584e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:02.989047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38b8fc-33d7-4a9a-b0e1-7a8eb034c754 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.989151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a34a0-703a-48a6-8f4b-df878a584e6b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.989228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38b8fc-33d7-4a9a-b0e1-7a8eb034c754 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.989299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a34a0-703a-48a6-8f4b-df878a584e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.989395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38b8fc-33d7-4a9a-b0e1-7a8eb034c754 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.989480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a34a0-703a-48a6-8f4b-df878a584e6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.989553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 377/733] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:02.990706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f31f0-0665-4c73-92a0-490323928af7 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.990994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f31f0-0665-4c73-92a0-490323928af7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.991278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f31f0-0665-4c73-92a0-490323928af7 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.992742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 378/734] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:02.993692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b93ccf-9a70-4f0d-81b8-9a821e7d986f None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:02.993983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b93ccf-9a70-4f0d-81b8-9a821e7d986f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:02.994270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b93ccf-9a70-4f0d-81b8-9a821e7d986f None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:02.995717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 379/735] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:02.996521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b93ccf-9a70-4f0d-81b8-9a821e7d986f None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:03.004411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a34a0-703a-48a6-8f4b-df878a584e6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:03.005852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a34a0-703a-48a6-8f4b-df878a584e6b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-fa3a34a0-703a-48a6-8f4b-df878a584e6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:03.021190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:03.022304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:03.023367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:03.023367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:03.023367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:03.023367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:03.023367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 357/736] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:03.023945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6ac0-d724-4fde-9b3a-6278e52b6c6e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:03.024240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6ac0-d724-4fde-9b3a-6278e52b6c6e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:03.024504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6ac0-d724-4fde-9b3a-6278e52b6c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:03.044908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e01d6ac0-d724-4fde-9b3a-6278e52b6c6e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:03.046172 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e01d6ac0-d724-4fde-9b3a-6278e52b6c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:03.117461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 358/737] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:10:03.118734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8882fe6-3134-4193-bcee-7692cb569762 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:03.119137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8882fe6-3134-4193-bcee-7692cb569762 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:03.119308 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8882fe6-3134-4193-bcee-7692cb569762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:03.125546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b93ccf-9a70-4f0d-81b8-9a821e7d986f None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:03.139711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d8882fe6-3134-4193-bcee-7692cb569762 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:03.140911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d8882fe6-3134-4193-bcee-7692cb569762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:03.164718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fbde0-c7c6-4e79-ace1-6a925aae5a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:03.165397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fbde0-c7c6-4e79-ace1-6a925aae5a94 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:03.165686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fbde0-c7c6-4e79-ace1-6a925aae5a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:03.165970 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fbde0-c7c6-4e79-ace1-6a925aae5a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:03.176532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fbde0-c7c6-4e79-ace1-6a925aae5a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:03.178030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fbde0-c7c6-4e79-ace1-6a925aae5a94 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-388fbde0-c7c6-4e79-ace1-6a925aae5a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:03.192726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:03.193887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:03.194847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:03.194847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:03.194847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:03.194847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:03.195244 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:03.198576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 380/738] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:03.200024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ae862-6f57-40f2-883c-630f68bb0561 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:03.200258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ae862-6f57-40f2-883c-630f68bb0561 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:03.200541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ae862-6f57-40f2-883c-630f68bb0561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:03.209859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 359/739] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:10:03.210697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8882fe6-3134-4193-bcee-7692cb569762 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:03.218897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-670ae862-6f57-40f2-883c-630f68bb0561 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:03.220109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-670ae862-6f57-40f2-883c-630f68bb0561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:03.264457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:03.293566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 381/740] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:10:03.294403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ae862-6f57-40f2-883c-630f68bb0561 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:03.326003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8882fe6-3134-4193-bcee-7692cb569762 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:03.364298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa81c2-217c-447c-a87d-2ceba7afbfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:03.364935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa81c2-217c-447c-a87d-2ceba7afbfbf None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:03.365226 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa81c2-217c-447c-a87d-2ceba7afbfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:03.365570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa81c2-217c-447c-a87d-2ceba7afbfbf None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:03.366232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa81c2-217c-447c-a87d-2ceba7afbfbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:03.366541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa81c2-217c-447c-a87d-2ceba7afbfbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:03.368043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa81c2-217c-447c-a87d-2ceba7afbfbf None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:03.385121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 360/741] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:03.386009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa81c2-217c-447c-a87d-2ceba7afbfbf None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:03.403147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ae862-6f57-40f2-883c-630f68bb0561 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:03.437877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3ec67-9a92-4cb6-b26e-b562099b3db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:03.438501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3ec67-9a92-4cb6-b26e-b562099b3db9 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:03.438790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3ec67-9a92-4cb6-b26e-b562099b3db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:03.439123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3ec67-9a92-4cb6-b26e-b562099b3db9 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:03.439578 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:03.440021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3ec67-9a92-4cb6-b26e-b562099b3db9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:03.441440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3ec67-9a92-4cb6-b26e-b562099b3db9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:03.490751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa81c2-217c-447c-a87d-2ceba7afbfbf None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:03.524303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fce669-1b8a-49a0-b2a7-9fdda0f6510e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:03.524852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fce669-1b8a-49a0-b2a7-9fdda0f6510e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:03.525140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fce669-1b8a-49a0-b2a7-9fdda0f6510e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:03.525453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fce669-1b8a-49a0-b2a7-9fdda0f6510e None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:03.526147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fce669-1b8a-49a0-b2a7-9fdda0f6510e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:03.526455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fce669-1b8a-49a0-b2a7-9fdda0f6510e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:03.527832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fce669-1b8a-49a0-b2a7-9fdda0f6510e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:03.538381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 382/742] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:10:03 2022] POST /identity/v3/services => generated 251 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:03.539218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3ec67-9a92-4cb6-b26e-b562099b3db9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:03.555877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 361/743] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:03.555877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fce669-1b8a-49a0-b2a7-9fdda0f6510e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:03.585506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:03.667718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fce669-1b8a-49a0-b2a7-9fdda0f6510e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:03.674059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3ec67-9a92-4cb6-b26e-b562099b3db9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:03.679269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:03.711011 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9d797-8586-4531-b13e-ce46f92cc808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:03.711643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9d797-8586-4531-b13e-ce46f92cc808 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:03.711932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9d797-8586-4531-b13e-ce46f92cc808 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:03.712210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9d797-8586-4531-b13e-ce46f92cc808 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:03.712923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9d797-8586-4531-b13e-ce46f92cc808 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:03.714403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9d797-8586-4531-b13e-ce46f92cc808 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:03.718159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204eb48-cd28-43cf-bca1-c7e556d3e12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:03.719042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204eb48-cd28-43cf-bca1-c7e556d3e12a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:03.719332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204eb48-cd28-43cf-bca1-c7e556d3e12a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:03.719620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204eb48-cd28-43cf-bca1-c7e556d3e12a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:03.730721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204eb48-cd28-43cf-bca1-c7e556d3e12a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-c204eb48-cd28-43cf-bca1-c7e556d3e12a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:03.735323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:03.736409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:03.737403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 383/744] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:03.738280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204eb48-cd28-43cf-bca1-c7e556d3e12a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:03.812324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 362/745] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:10:03 2022] POST /identity/v3/services => generated 240 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:03.813464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9d797-8586-4531-b13e-ce46f92cc808 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:03.842393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:03.858272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204eb48-cd28-43cf-bca1-c7e556d3e12a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:03.912015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146e02-e1d5-4dd6-9af2-c339c12655ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:03.913107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146e02-e1d5-4dd6-9af2-c339c12655ea None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:03.913613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146e02-e1d5-4dd6-9af2-c339c12655ea None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:03.914035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146e02-e1d5-4dd6-9af2-c339c12655ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:03.923236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b146e02-e1d5-4dd6-9af2-c339c12655ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:03.926038 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b146e02-e1d5-4dd6-9af2-c339c12655ea None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-9b146e02-e1d5-4dd6-9af2-c339c12655ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:03.943659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:03.944921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:03.946073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:03.946073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:03.946073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:03.946073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:03.946073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:03.946073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:03.946073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:03.946073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:03.946073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:03.946073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:03.948641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 384/746] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 28 06:10:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:03.949734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146e02-e1d5-4dd6-9af2-c339c12655ea None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:03.966484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9d797-8586-4531-b13e-ce46f92cc808 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:04.013022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08760fa-7322-44e0-8f43-8689d4d0577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:04.013855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08760fa-7322-44e0-8f43-8689d4d0577f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:04.014362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08760fa-7322-44e0-8f43-8689d4d0577f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:04.014820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08760fa-7322-44e0-8f43-8689d4d0577f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:04.023867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08760fa-7322-44e0-8f43-8689d4d0577f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:04.025554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08760fa-7322-44e0-8f43-8689d4d0577f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a08760fa-7322-44e0-8f43-8689d4d0577f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:04.040938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:04.042206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:04.043230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:04.043230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:04.043230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:04.043230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:04.043230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:04.043230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:04.043230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:04.043230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:04.043230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 363/747] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:04.046004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08760fa-7322-44e0-8f43-8689d4d0577f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:04.073322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146e02-e1d5-4dd6-9af2-c339c12655ea None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:04.086136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:04.122748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e78af-3b6a-4cc0-9401-e8a7beb0e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:04.123378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e78af-3b6a-4cc0-9401-e8a7beb0e3df None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:04.123661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e78af-3b6a-4cc0-9401-e8a7beb0e3df None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:04.123937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e78af-3b6a-4cc0-9401-e8a7beb0e3df None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:04.124669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e78af-3b6a-4cc0-9401-e8a7beb0e3df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:04.125142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e78af-3b6a-4cc0-9401-e8a7beb0e3df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:04.126475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e78af-3b6a-4cc0-9401-e8a7beb0e3df None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:04.137019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 385/748] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:10:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:04.137912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e78af-3b6a-4cc0-9401-e8a7beb0e3df None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:04.152437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:04.186840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08760fa-7322-44e0-8f43-8689d4d0577f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:04.234538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c60cbc-7fe5-4cad-8afc-a8eb8b313f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:04.235309 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60cbc-7fe5-4cad-8afc-a8eb8b313f1a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:04.235668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60cbc-7fe5-4cad-8afc-a8eb8b313f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:04.236584 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60cbc-7fe5-4cad-8afc-a8eb8b313f1a None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:04.237778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c60cbc-7fe5-4cad-8afc-a8eb8b313f1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:04.237949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c60cbc-7fe5-4cad-8afc-a8eb8b313f1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:04.239574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c60cbc-7fe5-4cad-8afc-a8eb8b313f1a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:04.257365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e78af-3b6a-4cc0-9401-e8a7beb0e3df None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:04.264090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 364/749] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 28 06:10:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:04.265116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c60cbc-7fe5-4cad-8afc-a8eb8b313f1a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:04.303764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac650125-30ca-4732-97d8-be44a7aabfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:04.304735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac650125-30ca-4732-97d8-be44a7aabfd3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:04.305210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac650125-30ca-4732-97d8-be44a7aabfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:04.305734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac650125-30ca-4732-97d8-be44a7aabfd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:04.306687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac650125-30ca-4732-97d8-be44a7aabfd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:04.307188 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac650125-30ca-4732-97d8-be44a7aabfd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:04.309292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac650125-30ca-4732-97d8-be44a7aabfd3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:04.373874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:04.379461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c60cbc-7fe5-4cad-8afc-a8eb8b313f1a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:04.426858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8d3e0-7218-41ca-88dc-1fa0525d75b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:04.427642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8d3e0-7218-41ca-88dc-1fa0525d75b9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:04.428002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8d3e0-7218-41ca-88dc-1fa0525d75b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:04.428357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8d3e0-7218-41ca-88dc-1fa0525d75b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:04.440784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8d3e0-7218-41ca-88dc-1fa0525d75b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:04.442316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8d3e0-7218-41ca-88dc-1fa0525d75b9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-81e8d3e0-7218-41ca-88dc-1fa0525d75b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:04.471921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:04.476598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:04.477529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:04.477529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:04.477529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:04.477529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:04.477529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 365/750] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:04.477529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8d3e0-7218-41ca-88dc-1fa0525d75b9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:04.479689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 386/751] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:04.480529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac650125-30ca-4732-97d8-be44a7aabfd3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:04.575915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:04.614342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8d3e0-7218-41ca-88dc-1fa0525d75b9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:04.615302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac650125-30ca-4732-97d8-be44a7aabfd3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:04.663879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8c70c-a57f-4368-a096-39d89db65b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:04.664249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc80ac2-0735-41e9-9c2a-aef676d96bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:04.664912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c70c-a57f-4368-a096-39d89db65b05 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:04.665046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc80ac2-0735-41e9-9c2a-aef676d96bed None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:04.665469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc80ac2-0735-41e9-9c2a-aef676d96bed None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:04.665626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c70c-a57f-4368-a096-39d89db65b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:04.665954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc80ac2-0735-41e9-9c2a-aef676d96bed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:04.666196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c70c-a57f-4368-a096-39d89db65b05 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:04.667146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c70c-a57f-4368-a096-39d89db65b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:04.667593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c70c-a57f-4368-a096-39d89db65b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:04.669455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c70c-a57f-4368-a096-39d89db65b05 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:04.677681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:04.687426 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 387/752] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:04.689465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8c70c-a57f-4368-a096-39d89db65b05 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:04.759115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc80ac2-0735-41e9-9c2a-aef676d96bed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:04.762008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc80ac2-0735-41e9-9c2a-aef676d96bed None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:04.812739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8c70c-a57f-4368-a096-39d89db65b05 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:04.845993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:04.852372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189264b3-c991-4884-ac27-1c6d3a505e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:04.853119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189264b3-c991-4884-ac27-1c6d3a505e09 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:04.853538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189264b3-c991-4884-ac27-1c6d3a505e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:04.853928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189264b3-c991-4884-ac27-1c6d3a505e09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:04.854838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189264b3-c991-4884-ac27-1c6d3a505e09 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=77571de30e934d6590ff7b58a000f5e2) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:04.855258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189264b3-c991-4884-ac27-1c6d3a505e09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:04.856884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189264b3-c991-4884-ac27-1c6d3a505e09 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:04.870173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 388/753] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:04 2022] GET /identity/v3/role_assignments?user.id=77571de30e934d6590ff7b58a000f5e2&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:04.871309 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189264b3-c991-4884-ac27-1c6d3a505e09 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:04.902689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 366/754] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:10:04.910254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc80ac2-0735-41e9-9c2a-aef676d96bed None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:04.960435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:05.071290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc80ac2-0735-41e9-9c2a-aef676d96bed None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:05.114376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189264b3-c991-4884-ac27-1c6d3a505e09 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:05.124880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593c55b-2efe-43d0-985e-d92d12df024e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:05.126139 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593c55b-2efe-43d0-985e-d92d12df024e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:05.126706 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593c55b-2efe-43d0-985e-d92d12df024e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:05.127092 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593c55b-2efe-43d0-985e-d92d12df024e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:05.143688 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593c55b-2efe-43d0-985e-d92d12df024e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:05.146105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593c55b-2efe-43d0-985e-d92d12df024e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:05.168893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaef01-7e9b-4952-a47a-9b878c12c985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:05.169587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaef01-7e9b-4952-a47a-9b878c12c985 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:05.169876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaef01-7e9b-4952-a47a-9b878c12c985 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:05.170154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaef01-7e9b-4952-a47a-9b878c12c985 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7593c55b-2efe-43d0-985e-d92d12df024e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:05.176636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:05.177555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:05.178428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:05.178428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:05.178428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:05.178428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:05.184842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 367/755] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:05.185761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593c55b-2efe-43d0-985e-d92d12df024e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:05.190202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaef01-7e9b-4952-a47a-9b878c12c985 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:05.191680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaef01-7e9b-4952-a47a-9b878c12c985 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-e5eaef01-7e9b-4952-a47a-9b878c12c985 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:05.215927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:05.216892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:05.218733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:05.218733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:05.218733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:05.218733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:05.218733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:05.218733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:05.218733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:05.218733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:05.218733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:10:05.218733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:05.223083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 389/756] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 28 06:10:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:05.224263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaef01-7e9b-4952-a47a-9b878c12c985 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:05.357346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:05.357547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:05.369482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593c55b-2efe-43d0-985e-d92d12df024e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:05.411564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c6fbb-7c0f-48a7-9006-a80c299b753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:05.412565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6fbb-7c0f-48a7-9006-a80c299b753a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:05.414606 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6fbb-7c0f-48a7-9006-a80c299b753a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:05.414907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6fbb-7c0f-48a7-9006-a80c299b753a None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:05.415634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c6fbb-7c0f-48a7-9006-a80c299b753a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:05.415953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c6fbb-7c0f-48a7-9006-a80c299b753a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:05.417371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c6fbb-7c0f-48a7-9006-a80c299b753a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:05.446439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 368/757] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:05.450265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c6fbb-7c0f-48a7-9006-a80c299b753a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:05.480951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaef01-7e9b-4952-a47a-9b878c12c985 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:05.523729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f67158-3e35-4e4b-a6b7-541616fe05df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:05.524729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f67158-3e35-4e4b-a6b7-541616fe05df None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:05.525270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f67158-3e35-4e4b-a6b7-541616fe05df None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:05.525824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f67158-3e35-4e4b-a6b7-541616fe05df None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:05.526822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f67158-3e35-4e4b-a6b7-541616fe05df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:05.527330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f67158-3e35-4e4b-a6b7-541616fe05df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:05.528963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f67158-3e35-4e4b-a6b7-541616fe05df None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:05.556552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 390/758] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 28 06:10:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:05.562236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f67158-3e35-4e4b-a6b7-541616fe05df None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:05.673111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c6fbb-7c0f-48a7-9006-a80c299b753a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:05.737209 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f67158-3e35-4e4b-a6b7-541616fe05df None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:05.744880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c93f8-e503-4839-9082-cb49e30d51ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:05.745583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c93f8-e503-4839-9082-cb49e30d51ad None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:05.745874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c93f8-e503-4839-9082-cb49e30d51ad None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:05.746158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c93f8-e503-4839-9082-cb49e30d51ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:05.753866 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c93f8-e503-4839-9082-cb49e30d51ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:05.755262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:05.755377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c93f8-e503-4839-9082-cb49e30d51ad None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2e3c93f8-e503-4839-9082-cb49e30d51ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:05.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:05.769174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:05.770105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:05.770105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:05.770105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:05.770105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:05.770105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:05.770105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:05.770105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:05.770105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:05.773905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 369/759] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:05.775227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377454b-daeb-4054-91fc-e2b5984564fd None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:05.775524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377454b-daeb-4054-91fc-e2b5984564fd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:05.775810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377454b-daeb-4054-91fc-e2b5984564fd None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:05.777404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 370/760] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:05.778129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377454b-daeb-4054-91fc-e2b5984564fd None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:05.784502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b8e74-b05a-448c-baee-eea03fb72532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:05.785589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b8e74-b05a-448c-baee-eea03fb72532 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:05.786135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b8e74-b05a-448c-baee-eea03fb72532 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:05.786577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b8e74-b05a-448c-baee-eea03fb72532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:05.796364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:05.799922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b8e74-b05a-448c-baee-eea03fb72532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:05.801659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b8e74-b05a-448c-baee-eea03fb72532 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-171b8e74-b05a-448c-baee-eea03fb72532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:05.819508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:05.820432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:05.821360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:05.821360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:05.821360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:05.821360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:05.822939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 391/761] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:05.824169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dd8f3-0995-493e-906c-f480d5678fb8 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:05.824461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dd8f3-0995-493e-906c-f480d5678fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:05.824917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dd8f3-0995-493e-906c-f480d5678fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:05.846779 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-3d4dd8f3-0995-493e-906c-f480d5678fb8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:05.848060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3d4dd8f3-0995-493e-906c-f480d5678fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:05.915240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377454b-daeb-4054-91fc-e2b5984564fd None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:05.929725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 392/762] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:05.930637 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4dd8f3-0995-493e-906c-f480d5678fb8 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:05.958223 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2f030-6dc1-45b2-a26e-9d6ceb46ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:05.959467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2f030-6dc1-45b2-a26e-9d6ceb46ef80 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:05.959963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2f030-6dc1-45b2-a26e-9d6ceb46ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:05.960860 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2f030-6dc1-45b2-a26e-9d6ceb46ef80 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:05.961995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2f030-6dc1-45b2-a26e-9d6ceb46ef80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:05.962510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2f030-6dc1-45b2-a26e-9d6ceb46ef80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:05.964566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2f030-6dc1-45b2-a26e-9d6ceb46ef80 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:05.982908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 371/763] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:05.984059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2f030-6dc1-45b2-a26e-9d6ceb46ef80 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:06.060104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:06.061235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:06.067549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4dd8f3-0995-493e-906c-f480d5678fb8 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:06.094899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2f030-6dc1-45b2-a26e-9d6ceb46ef80 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:06.108190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f81e61-521b-43f3-b7ec-b274e38b6ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:06.108996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f81e61-521b-43f3-b7ec-b274e38b6ffd None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.109279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f81e61-521b-43f3-b7ec-b274e38b6ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.109596 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f81e61-521b-43f3-b7ec-b274e38b6ffd None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.110308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f81e61-521b-43f3-b7ec-b274e38b6ffd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:06.110638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f81e61-521b-43f3-b7ec-b274e38b6ffd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:06.112175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f81e61-521b-43f3-b7ec-b274e38b6ffd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:06.124566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 393/764] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:06.125712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f81e61-521b-43f3-b7ec-b274e38b6ffd None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:06.141583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca1858-687f-4f08-b3e3-e68e5b282f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:06.142814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca1858-687f-4f08-b3e3-e68e5b282f1a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.143118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca1858-687f-4f08-b3e3-e68e5b282f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.143430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca1858-687f-4f08-b3e3-e68e5b282f1a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.144374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca1858-687f-4f08-b3e3-e68e5b282f1a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:06.145524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca1858-687f-4f08-b3e3-e68e5b282f1a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b2ca1858-687f-4f08-b3e3-e68e5b282f1a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:06.154760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:06.155762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:06.156649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 372/765] 10.222.0.69 () {56 vars in 1129 bytes} [Thu Jul 28 06:10:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:06.158204 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da0676-4967-41df-848f-69c09b8258ad None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.158204 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da0676-4967-41df-848f-69c09b8258ad None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.158204 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da0676-4967-41df-848f-69c09b8258ad None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.161443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 373/766] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:06.161443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da0676-4967-41df-848f-69c09b8258ad None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:06.246616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f81e61-521b-43f3-b7ec-b274e38b6ffd None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:06.276925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da0676-4967-41df-848f-69c09b8258ad None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:06.284466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152a6e2-fb71-4703-bf92-6fe7eab8d647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:06.285093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152a6e2-fb71-4703-bf92-6fe7eab8d647 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.285456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152a6e2-fb71-4703-bf92-6fe7eab8d647 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.285742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152a6e2-fb71-4703-bf92-6fe7eab8d647 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.286547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152a6e2-fb71-4703-bf92-6fe7eab8d647 None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=39c6f0ee9b78413eb8c5efc7adf145b6) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:06.286863 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152a6e2-fb71-4703-bf92-6fe7eab8d647 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:06.288303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152a6e2-fb71-4703-bf92-6fe7eab8d647 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:06.294046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 394/767] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:06 2022] GET /identity/v3/role_assignments?user.id=39c6f0ee9b78413eb8c5efc7adf145b6&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:06.294840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152a6e2-fb71-4703-bf92-6fe7eab8d647 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:06.331261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:06.332708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f9b58-f3ed-406d-a441-ca35ac4ba45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:06.333641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f9b58-f3ed-406d-a441-ca35ac4ba45a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.334121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f9b58-f3ed-406d-a441-ca35ac4ba45a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.335644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f9b58-f3ed-406d-a441-ca35ac4ba45a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.338596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:06.347378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f9b58-f3ed-406d-a441-ca35ac4ba45a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:06.349342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f9b58-f3ed-406d-a441-ca35ac4ba45a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-848f9b58-f3ed-406d-a441-ca35ac4ba45a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:06.366236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:06.367213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:06.368105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:06.368105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:06.368105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:06.368105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:06.368105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:06.368105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:06.368105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:06.368105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:06.370709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 374/768] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:06.372824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d6883-e2da-46b8-b77c-6d19cc803cd8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.373249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d6883-e2da-46b8-b77c-6d19cc803cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.373686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d6883-e2da-46b8-b77c-6d19cc803cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.401421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-db5d6883-e2da-46b8-b77c-6d19cc803cd8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:06.403011 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-db5d6883-e2da-46b8-b77c-6d19cc803cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:06.438466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152a6e2-fb71-4703-bf92-6fe7eab8d647 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:06.479387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7622c83-eceb-4d8e-9ea1-db3231b446f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:06.480178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7622c83-eceb-4d8e-9ea1-db3231b446f9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.480568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7622c83-eceb-4d8e-9ea1-db3231b446f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.481112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7622c83-eceb-4d8e-9ea1-db3231b446f9 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.482614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7622c83-eceb-4d8e-9ea1-db3231b446f9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:06.482614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7622c83-eceb-4d8e-9ea1-db3231b446f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:06.484420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7622c83-eceb-4d8e-9ea1-db3231b446f9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:06.491817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 395/769] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 28 06:10:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:06.495914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7622c83-eceb-4d8e-9ea1-db3231b446f9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:06.496178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 375/770] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:06.497262 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d6883-e2da-46b8-b77c-6d19cc803cd8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:06.605846 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:06.623783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d6883-e2da-46b8-b77c-6d19cc803cd8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:06.623972 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7622c83-eceb-4d8e-9ea1-db3231b446f9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:06.654705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:06.670890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a293e45-e4c4-4970-83a2-603268b61fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:06.671613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a293e45-e4c4-4970-83a2-603268b61fff None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.672033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a293e45-e4c4-4970-83a2-603268b61fff None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.672409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a293e45-e4c4-4970-83a2-603268b61fff None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.673269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a293e45-e4c4-4970-83a2-603268b61fff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:06.673524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc3fe0-1a5d-4d5e-96f4-23af9f0d2b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:06.673982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a293e45-e4c4-4970-83a2-603268b61fff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:06.674932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc3fe0-1a5d-4d5e-96f4-23af9f0d2b18 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.675255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a293e45-e4c4-4970-83a2-603268b61fff None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:06.675589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc3fe0-1a5d-4d5e-96f4-23af9f0d2b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.676056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc3fe0-1a5d-4d5e-96f4-23af9f0d2b18 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.676949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc3fe0-1a5d-4d5e-96f4-23af9f0d2b18 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:06.677704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc3fe0-1a5d-4d5e-96f4-23af9f0d2b18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:06.679879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc3fe0-1a5d-4d5e-96f4-23af9f0d2b18 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:06.687132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 376/771] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 28 06:10:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:06.688570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc3fe0-1a5d-4d5e-96f4-23af9f0d2b18 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:06.693404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 396/772] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:06.694449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a293e45-e4c4-4970-83a2-603268b61fff None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:06.820943 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc3fe0-1a5d-4d5e-96f4-23af9f0d2b18 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:06.824574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a293e45-e4c4-4970-83a2-603268b61fff None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:06.857971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e47e65-a66d-49ee-a25c-0697e8bb41fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:06.858689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e47e65-a66d-49ee-a25c-0697e8bb41fe None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.859083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e47e65-a66d-49ee-a25c-0697e8bb41fe None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.859469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e47e65-a66d-49ee-a25c-0697e8bb41fe None admin] PATH_INFO: `/v3/services/network` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.860379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e47e65-a66d-49ee-a25c-0697e8bb41fe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:06.862909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e47e65-a66d-49ee-a25c-0697e8bb41fe None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:06.870648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d8d4a-9683-4c60-ba9a-37379b9ed0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:06.871482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d8d4a-9683-4c60-ba9a-37379b9ed0ec None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:06.871937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d8d4a-9683-4c60-ba9a-37379b9ed0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:06.872392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d8d4a-9683-4c60-ba9a-37379b9ed0ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-16e47e65-a66d-49ee-a25c-0697e8bb41fe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:06.872983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:06.874180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:10:06.875224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:06.876843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d8d4a-9683-4c60-ba9a-37379b9ed0ec None admin] RBAC: Adding query filter params (interface=public, service_id=f3f6f4fb48384fb4a6c3d19670424600, region_id=RegionOne) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:06.881949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 377/773] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:06 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:06.881949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e47e65-a66d-49ee-a25c-0697e8bb41fe None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:06.882144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d8d4a-9683-4c60-ba9a-37379b9ed0ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:06.883075 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d8d4a-9683-4c60-ba9a-37379b9ed0ec None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:06.896121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 397/774] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 28 06:10:06 2022] GET /identity/v3/endpoints?service_id=f3f6f4fb48384fb4a6c3d19670424600&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:06.897648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d8d4a-9683-4c60-ba9a-37379b9ed0ec None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:06.909975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:06.943119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:06.994518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e47e65-a66d-49ee-a25c-0697e8bb41fe None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:07.032965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d8d4a-9683-4c60-ba9a-37379b9ed0ec None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:07.041267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9f0bd-00ab-4bd0-b659-4104fd3fe5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:07.042287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9f0bd-00ab-4bd0-b659-4104fd3fe5f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:07.043234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9f0bd-00ab-4bd0-b659-4104fd3fe5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:07.043726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9f0bd-00ab-4bd0-b659-4104fd3fe5f9 None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/5706aa2d6bf54f05badf9588f51f4677/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:07.046686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9f0bd-00ab-4bd0-b659-4104fd3fe5f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=5706aa2d6bf54f05badf9588f51f4677, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:07.049971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9f0bd-00ab-4bd0-b659-4104fd3fe5f9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:07.070826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb715581-05b3-4147-af90-15c4ddc3e6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:07.071454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb715581-05b3-4147-af90-15c4ddc3e6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:07.071793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb715581-05b3-4147-af90-15c4ddc3e6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:07.072076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb715581-05b3-4147-af90-15c4ddc3e6a3 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:07.072797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb715581-05b3-4147-af90-15c4ddc3e6a3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:07.073110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb715581-05b3-4147-af90-15c4ddc3e6a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:07.074563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb715581-05b3-4147-af90-15c4ddc3e6a3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:07.082994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 398/775] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:07.082994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb715581-05b3-4147-af90-15c4ddc3e6a3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:07.120522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 378/776] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:10:06 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/5706aa2d6bf54f05badf9588f51f4677/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:10:07.130911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457fb5e-d001-4c1c-beb9-e354b56eeadc None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:07.131244 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457fb5e-d001-4c1c-beb9-e354b56eeadc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:07.131517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457fb5e-d001-4c1c-beb9-e354b56eeadc None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:07.133086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 379/777] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:07.133636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457fb5e-d001-4c1c-beb9-e354b56eeadc None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:07.169409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:07.216862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb715581-05b3-4147-af90-15c4ddc3e6a3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:07.260755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba0033-f0e7-42a6-9b76-c2c8ffe686b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:07.261833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba0033-f0e7-42a6-9b76-c2c8ffe686b2 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:07.262198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba0033-f0e7-42a6-9b76-c2c8ffe686b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:07.262591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba0033-f0e7-42a6-9b76-c2c8ffe686b2 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:07.263418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba0033-f0e7-42a6-9b76-c2c8ffe686b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:07.264906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba0033-f0e7-42a6-9b76-c2c8ffe686b2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:07.271516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 399/778] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 28 06:10:07 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:07.273088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef1e1-01e7-4993-910a-ce74a069af45 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:07.273497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef1e1-01e7-4993-910a-ce74a069af45 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:07.273860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef1e1-01e7-4993-910a-ce74a069af45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:07.292961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457fb5e-d001-4c1c-beb9-e354b56eeadc None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:07.300887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f7aef1e1-01e7-4993-910a-ce74a069af45 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:07.301091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f7aef1e1-01e7-4993-910a-ce74a069af45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:07.321804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:07.359193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b360a-7183-4f2c-a52a-14303a8d7b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:07.360011 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b360a-7183-4f2c-a52a-14303a8d7b14 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:07.360372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b360a-7183-4f2c-a52a-14303a8d7b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:07.361571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b360a-7183-4f2c-a52a-14303a8d7b14 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:07.362488 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b360a-7183-4f2c-a52a-14303a8d7b14 None admin] RBAC: Adding query filter params (type=network) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:07.362875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b360a-7183-4f2c-a52a-14303a8d7b14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:07.364661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b360a-7183-4f2c-a52a-14303a8d7b14 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:07.372188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 380/779] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:07.378545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b360a-7183-4f2c-a52a-14303a8d7b14 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:07.408804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 400/780] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:07.477352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef1e1-01e7-4993-910a-ce74a069af45 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:07.528098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b360a-7183-4f2c-a52a-14303a8d7b14 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:07.569791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:07.574709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81976f-93bb-493b-9ea5-f5c069447bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:07.577033 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81976f-93bb-493b-9ea5-f5c069447bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:07.577682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81976f-93bb-493b-9ea5-f5c069447bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:07.578175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81976f-93bb-493b-9ea5-f5c069447bd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:07.579245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81976f-93bb-493b-9ea5-f5c069447bd9 None admin] RBAC: Adding query filter params (interface=public, service_id=b4f17521a7e045e1a7604cd588028653, region_id=RegionOne) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:07.580450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81976f-93bb-493b-9ea5-f5c069447bd9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:07.588821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81976f-93bb-493b-9ea5-f5c069447bd9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:07.591184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 381/781] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 28 06:10:07 2022] GET /identity/v3/endpoints?service_id=b4f17521a7e045e1a7604cd588028653&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:07.599324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81976f-93bb-493b-9ea5-f5c069447bd9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:07.619109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef1e1-01e7-4993-910a-ce74a069af45 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:07.656583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:07.662023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1c664-d9e7-4874-9406-21aae72b3693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:07.662346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1c664-d9e7-4874-9406-21aae72b3693 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:07.664096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1c664-d9e7-4874-9406-21aae72b3693 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:07.664096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1c664-d9e7-4874-9406-21aae72b3693 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:07.675867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1c664-d9e7-4874-9406-21aae72b3693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-8be1c664-d9e7-4874-9406-21aae72b3693 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:07.682068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:07.683004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:07.684269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 401/782] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:07.689869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1c664-d9e7-4874-9406-21aae72b3693 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:07.796368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1c664-d9e7-4874-9406-21aae72b3693 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:07.822634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:07.827303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81976f-93bb-493b-9ea5-f5c069447bd9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:07.834697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcdef6-d6ce-42e8-97f5-a6d2d5627cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:07.835342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcdef6-d6ce-42e8-97f5-a6d2d5627cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:07.835635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcdef6-d6ce-42e8-97f5-a6d2d5627cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:07.835917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcdef6-d6ce-42e8-97f5-a6d2d5627cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:07.839413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcdef6-d6ce-42e8-97f5-a6d2d5627cd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:07.839413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcdef6-d6ce-42e8-97f5-a6d2d5627cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:07.839413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcdef6-d6ce-42e8-97f5-a6d2d5627cd9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:07.848886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 402/783] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:10:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:07.854534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcdef6-d6ce-42e8-97f5-a6d2d5627cd9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:07.906754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56404268-9fd5-471c-9fcf-b7c9fe39a81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:07.907319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56404268-9fd5-471c-9fcf-b7c9fe39a81c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:07.908237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56404268-9fd5-471c-9fcf-b7c9fe39a81c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:07.908694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56404268-9fd5-471c-9fcf-b7c9fe39a81c None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:07.909408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56404268-9fd5-471c-9fcf-b7c9fe39a81c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:07.910817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56404268-9fd5-471c-9fcf-b7c9fe39a81c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:07.928257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 382/784] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 28 06:10:07 2022] GET /identity/v3/services => generated 772 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:07.966535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcdef6-d6ce-42e8-97f5-a6d2d5627cd9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:08.005911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132557d-cb16-4ca2-a8fd-63688f5ba2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:08.006622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132557d-cb16-4ca2-a8fd-63688f5ba2fa None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:08.006921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132557d-cb16-4ca2-a8fd-63688f5ba2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:08.007206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132557d-cb16-4ca2-a8fd-63688f5ba2fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:08.019585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132557d-cb16-4ca2-a8fd-63688f5ba2fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:08.021070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132557d-cb16-4ca2-a8fd-63688f5ba2fa None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-4132557d-cb16-4ca2-a8fd-63688f5ba2fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:08.038140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:08.039102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:08.039987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:08.039987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:08.039987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:08.039987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:08.042392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 403/785] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:08.047670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56404268-9fd5-471c-9fcf-b7c9fe39a81c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:08.183180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56404268-9fd5-471c-9fcf-b7c9fe39a81c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:08.187631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:08.244112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38353cbc-eb16-42ab-93f0-77b19009f346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:08.245149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38353cbc-eb16-42ab-93f0-77b19009f346 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:08.245681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38353cbc-eb16-42ab-93f0-77b19009f346 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:08.246171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38353cbc-eb16-42ab-93f0-77b19009f346 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:08.247427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38353cbc-eb16-42ab-93f0-77b19009f346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:08.247968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38353cbc-eb16-42ab-93f0-77b19009f346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:08.250505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38353cbc-eb16-42ab-93f0-77b19009f346 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:08.274565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 383/786] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:08.281063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132557d-cb16-4ca2-a8fd-63688f5ba2fa None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:08.326941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:08.421264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132557d-cb16-4ca2-a8fd-63688f5ba2fa None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:08.478741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6944c31-0129-4c19-a68e-e6ea0a217fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:08.479344 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6944c31-0129-4c19-a68e-e6ea0a217fdc None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:08.479640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6944c31-0129-4c19-a68e-e6ea0a217fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:08.479926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6944c31-0129-4c19-a68e-e6ea0a217fdc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:08.494800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6944c31-0129-4c19-a68e-e6ea0a217fdc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:08.496293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6944c31-0129-4c19-a68e-e6ea0a217fdc None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-d6944c31-0129-4c19-a68e-e6ea0a217fdc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:08.509212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:08.510232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:08.511183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:08.511183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:08.511183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:08.511183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:08.511183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:08.511183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:08.511183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:08.511183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:08.511183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:10:08.511183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:08.515485 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 404/787] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:08.523829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38353cbc-eb16-42ab-93f0-77b19009f346 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:08.570840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76557539-e91c-4ab8-8dd4-2869f03104e0 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:08.571175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76557539-e91c-4ab8-8dd4-2869f03104e0 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:08.571429 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76557539-e91c-4ab8-8dd4-2869f03104e0 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:08.572916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 405/788] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:08.578782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc4c56-c4be-4329-a91d-741e7e80640f None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:08.579105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc4c56-c4be-4329-a91d-741e7e80640f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:08.579915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc4c56-c4be-4329-a91d-741e7e80640f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:08.610151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2fdc4c56-c4be-4329-a91d-741e7e80640f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:08.612058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2fdc4c56-c4be-4329-a91d-741e7e80640f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:08.647044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38353cbc-eb16-42ab-93f0-77b19009f346 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:08.684359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91ad4f-9694-44df-abb7-e0dae9400fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:08.685105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ad4f-9694-44df-abb7-e0dae9400fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:08.685428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ad4f-9694-44df-abb7-e0dae9400fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:08.685838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ad4f-9694-44df-abb7-e0dae9400fe7 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:08.686649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91ad4f-9694-44df-abb7-e0dae9400fe7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:08.686966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91ad4f-9694-44df-abb7-e0dae9400fe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:08.688429 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91ad4f-9694-44df-abb7-e0dae9400fe7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:08.708012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 384/789] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 28 06:10:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:08.714958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91ad4f-9694-44df-abb7-e0dae9400fe7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:08.736595 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 406/790] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:08.752914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:08.773622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc4c56-c4be-4329-a91d-741e7e80640f None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:08.822665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91ad4f-9694-44df-abb7-e0dae9400fe7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:08.842378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:08.861581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bbed6-ac39-4196-b530-c9a2c5b61dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:08.862933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bbed6-ac39-4196-b530-c9a2c5b61dee None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:08.863542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bbed6-ac39-4196-b530-c9a2c5b61dee None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:08.863970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bbed6-ac39-4196-b530-c9a2c5b61dee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:08.875066 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69bbed6-ac39-4196-b530-c9a2c5b61dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:08.877690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69bbed6-ac39-4196-b530-c9a2c5b61dee None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b69bbed6-ac39-4196-b530-c9a2c5b61dee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:08.904639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:08.905797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:08.906886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:08.906886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:08.906886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:08.906886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:08.916961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 385/791] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:08.919243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bbed6-ac39-4196-b530-c9a2c5b61dee None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:09.010092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc4c56-c4be-4329-a91d-741e7e80640f None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:09.029683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:09.054119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bbed6-ac39-4196-b530-c9a2c5b61dee None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:09.091100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb78f-71b9-410e-8e8f-95c4446df615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:09.092114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb78f-71b9-410e-8e8f-95c4446df615 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:09.092624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb78f-71b9-410e-8e8f-95c4446df615 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:09.093208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb78f-71b9-410e-8e8f-95c4446df615 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:09.094140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fb78f-71b9-410e-8e8f-95c4446df615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:09.094597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fb78f-71b9-410e-8e8f-95c4446df615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:09.096940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fb78f-71b9-410e-8e8f-95c4446df615 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:09.114690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 386/792] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:09.116141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb78f-71b9-410e-8e8f-95c4446df615 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:09.122240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7347cc-61b3-4592-bed3-c46236db7676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:09.123070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7347cc-61b3-4592-bed3-c46236db7676 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:09.123476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7347cc-61b3-4592-bed3-c46236db7676 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:09.123857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7347cc-61b3-4592-bed3-c46236db7676 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:09.166079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7347cc-61b3-4592-bed3-c46236db7676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-4b7347cc-61b3-4592-bed3-c46236db7676 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:09.171306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:09.172416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:10:09.173603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:09.174715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 407/793] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:10:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:09.179870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7347cc-61b3-4592-bed3-c46236db7676 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:09.229160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb78f-71b9-410e-8e8f-95c4446df615 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:09.268857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c5683-acd6-408e-b645-d4ee37ffc144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:09.269880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c5683-acd6-408e-b645-d4ee37ffc144 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:09.270352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c5683-acd6-408e-b645-d4ee37ffc144 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:09.270812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c5683-acd6-408e-b645-d4ee37ffc144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:09.278791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c5683-acd6-408e-b645-d4ee37ffc144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:09.280625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c5683-acd6-408e-b645-d4ee37ffc144 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:09.281518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-9a9c5683-acd6-408e-b645-d4ee37ffc144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:09.300055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:09.301048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:09.302002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:09.302002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:09.302002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:09.302002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:09.302002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:09.302002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:09.302002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:09.302002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:09.309645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 387/794] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:09.311728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c5683-acd6-408e-b645-d4ee37ffc144 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:09.426820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c5683-acd6-408e-b645-d4ee37ffc144 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:09.468060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcce6ad-bd68-43d6-b655-7a5cc5bb9543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:09.469081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcce6ad-bd68-43d6-b655-7a5cc5bb9543 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:09.469577 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcce6ad-bd68-43d6-b655-7a5cc5bb9543 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:09.471324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcce6ad-bd68-43d6-b655-7a5cc5bb9543 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:09.472354 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcce6ad-bd68-43d6-b655-7a5cc5bb9543 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:09.472835 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcce6ad-bd68-43d6-b655-7a5cc5bb9543 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:09.474705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcce6ad-bd68-43d6-b655-7a5cc5bb9543 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:09.495201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 388/795] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:09.498076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcce6ad-bd68-43d6-b655-7a5cc5bb9543 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:09.523470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:09.565680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:09.587297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7347cc-61b3-4592-bed3-c46236db7676 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:09.609610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcce6ad-bd68-43d6-b655-7a5cc5bb9543 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:09.647672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928b591-756f-43bb-bebf-82af0ea46d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:09.648529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b591-756f-43bb-bebf-82af0ea46d6d None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:09.648912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b591-756f-43bb-bebf-82af0ea46d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:09.649299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b591-756f-43bb-bebf-82af0ea46d6d None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/77571de30e934d6590ff7b58a000f5e2/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:09.651763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928b591-756f-43bb-bebf-82af0ea46d6d None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=77571de30e934d6590ff7b58a000f5e2, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:09.653331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928b591-756f-43bb-bebf-82af0ea46d6d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:09.679310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6b1ff-9819-4f9b-8ef4-96917f700e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:09.679933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6b1ff-9819-4f9b-8ef4-96917f700e48 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:09.680221 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6b1ff-9819-4f9b-8ef4-96917f700e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:09.680488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6b1ff-9819-4f9b-8ef4-96917f700e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:09.681208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6b1ff-9819-4f9b-8ef4-96917f700e48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:09.681556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6b1ff-9819-4f9b-8ef4-96917f700e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:09.683076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6b1ff-9819-4f9b-8ef4-96917f700e48 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:09.693054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 408/796] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:10:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 513 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:09.698711 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6b1ff-9819-4f9b-8ef4-96917f700e48 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:09.745993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 389/797] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:10:09 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/77571de30e934d6590ff7b58a000f5e2/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:10:09.816934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6b1ff-9819-4f9b-8ef4-96917f700e48 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:09.855735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6445c-9251-4845-b569-f182ab755552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:09.856551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6445c-9251-4845-b569-f182ab755552 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:09.856741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6445c-9251-4845-b569-f182ab755552 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:09.857083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6445c-9251-4845-b569-f182ab755552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:09.869781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6445c-9251-4845-b569-f182ab755552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:09.871320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6445c-9251-4845-b569-f182ab755552 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-97d6445c-9251-4845-b569-f182ab755552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:09.893162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:09.894156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:09.896567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:09.896567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:09.896567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:09.896567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:09.896567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 409/798] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:09.900321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928b591-756f-43bb-bebf-82af0ea46d6d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:09.989540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cacde-01cb-408b-ad73-a8d13bb1b044 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:09.989883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cacde-01cb-408b-ad73-a8d13bb1b044 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:09.990114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cacde-01cb-408b-ad73-a8d13bb1b044 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:09.991468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 410/799] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:09.997451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cea8c-1413-4537-9a49-f944f420f287 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:09.997742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cea8c-1413-4537-9a49-f944f420f287 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:09.998026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cea8c-1413-4537-9a49-f944f420f287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.006651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:10.021942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-792cea8c-1413-4537-9a49-f944f420f287 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:10.023152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-792cea8c-1413-4537-9a49-f944f420f287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:10.034868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928b591-756f-43bb-bebf-82af0ea46d6d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:10.053693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:10.073887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4468b0-ce53-41a3-a588-4c2e332f4ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:10.074200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4468b0-ce53-41a3-a588-4c2e332f4ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.074673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4468b0-ce53-41a3-a588-4c2e332f4ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.075070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4468b0-ce53-41a3-a588-4c2e332f4ef6 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.075779 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4468b0-ce53-41a3-a588-4c2e332f4ef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:10.076090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4468b0-ce53-41a3-a588-4c2e332f4ef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:10.077623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4468b0-ce53-41a3-a588-4c2e332f4ef6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:10.091547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 390/800] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:10.093086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94e7b-7f20-4639-b79d-99b65f352245 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.093409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94e7b-7f20-4639-b79d-99b65f352245 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.093800 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94e7b-7f20-4639-b79d-99b65f352245 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.095643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 391/801] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:10.098391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94e7b-7f20-4639-b79d-99b65f352245 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:10.102204 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 411/802] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:10.103574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571564-b849-4c08-aa1f-ac84432ccadc None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.103869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571564-b849-4c08-aa1f-ac84432ccadc None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.104158 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571564-b849-4c08-aa1f-ac84432ccadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.123792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-9f571564-b849-4c08-aa1f-ac84432ccadc None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:10.125002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9f571564-b849-4c08-aa1f-ac84432ccadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:10.199235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 412/803] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:10.200109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f571564-b849-4c08-aa1f-ac84432ccadc None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:10.213884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94e7b-7f20-4639-b79d-99b65f352245 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:10.253392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9a9e0-def6-4326-b910-95a97ed5cbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:10.254293 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9a9e0-def6-4326-b910-95a97ed5cbed None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.254675 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9a9e0-def6-4326-b910-95a97ed5cbed None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.255091 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9a9e0-def6-4326-b910-95a97ed5cbed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.264653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9a9e0-def6-4326-b910-95a97ed5cbed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:10.265474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:10.268270 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9a9e0-def6-4326-b910-95a97ed5cbed None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-52b9a9e0-def6-4326-b910-95a97ed5cbed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:10.281448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:10.282587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:10.283523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:10.283523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:10.283523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:10.283523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:10.283523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:10.283523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:10.283523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:10.283523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:10.283523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:10.283523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:10.286238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 392/804] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 28 06:10:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:10.287343 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9a9e0-def6-4326-b910-95a97ed5cbed None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:10.307628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f571564-b849-4c08-aa1f-ac84432ccadc None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:10.327646 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:10.346786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc6815-74ce-485b-9a21-6b33b394b634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:10.347462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc6815-74ce-485b-9a21-6b33b394b634 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.347751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc6815-74ce-485b-9a21-6b33b394b634 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.348190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc6815-74ce-485b-9a21-6b33b394b634 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.348840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc6815-74ce-485b-9a21-6b33b394b634 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:10.350358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc6815-74ce-485b-9a21-6b33b394b634 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-36dc6815-74ce-485b-9a21-6b33b394b634 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:10.359852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:10.361685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:10.362788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 413/805] 10.222.0.69 () {56 vars in 1129 bytes} [Thu Jul 28 06:10:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:10.363894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc6815-74ce-485b-9a21-6b33b394b634 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:10.411662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9a9e0-def6-4326-b910-95a97ed5cbed None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:10.457073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7acb8-a039-4e82-97f1-df2fb7e5b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:10.457857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7acb8-a039-4e82-97f1-df2fb7e5b311 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.458148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7acb8-a039-4e82-97f1-df2fb7e5b311 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.458430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7acb8-a039-4e82-97f1-df2fb7e5b311 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.470789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7acb8-a039-4e82-97f1-df2fb7e5b311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-01c7acb8-a039-4e82-97f1-df2fb7e5b311 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:10.476966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:10.481038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 393/806] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:10.481984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7acb8-a039-4e82-97f1-df2fb7e5b311 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:10.492395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc6815-74ce-485b-9a21-6b33b394b634 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:10.520802 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:10.529362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f4b4f-66df-44aa-8622-609b29189b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:10.530086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f4b4f-66df-44aa-8622-609b29189b90 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.530444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f4b4f-66df-44aa-8622-609b29189b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.530726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f4b4f-66df-44aa-8622-609b29189b90 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.531922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f4b4f-66df-44aa-8622-609b29189b90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:10.532764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f4b4f-66df-44aa-8622-609b29189b90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:10.534407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f4b4f-66df-44aa-8622-609b29189b90 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:10.557843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 414/807] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 28 06:10:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:10.557843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af0eb0-c14f-4786-aab4-17b958da430f None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.557843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af0eb0-c14f-4786-aab4-17b958da430f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.557843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af0eb0-c14f-4786-aab4-17b958da430f None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.558500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 415/808] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:10.561413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af0eb0-c14f-4786-aab4-17b958da430f None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:10.581454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:10.588056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7acb8-a039-4e82-97f1-df2fb7e5b311 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:10.625952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cae6b-e738-4a43-b7b5-a2d415c8db08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:10.626958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cae6b-e738-4a43-b7b5-a2d415c8db08 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.627341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cae6b-e738-4a43-b7b5-a2d415c8db08 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.627730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cae6b-e738-4a43-b7b5-a2d415c8db08 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.628506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724cae6b-e738-4a43-b7b5-a2d415c8db08 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:10.628915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724cae6b-e738-4a43-b7b5-a2d415c8db08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:10.630439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724cae6b-e738-4a43-b7b5-a2d415c8db08 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:10.640151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 394/809] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 28 06:10:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:10.640151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cae6b-e738-4a43-b7b5-a2d415c8db08 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:10.685097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af0eb0-c14f-4786-aab4-17b958da430f None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:10.721865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703218e-c632-45b1-9301-2be71bee1733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:10.722597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703218e-c632-45b1-9301-2be71bee1733 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.722948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703218e-c632-45b1-9301-2be71bee1733 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.723296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703218e-c632-45b1-9301-2be71bee1733 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.724087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703218e-c632-45b1-9301-2be71bee1733 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:10.724460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703218e-c632-45b1-9301-2be71bee1733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:10.725955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703218e-c632-45b1-9301-2be71bee1733 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:10.739613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 416/810] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:10:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:10.739613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8f594-9388-4ff1-80f4-56129b34f89d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.739613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8f594-9388-4ff1-80f4-56129b34f89d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.739613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8f594-9388-4ff1-80f4-56129b34f89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.749783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cae6b-e738-4a43-b7b5-a2d415c8db08 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:10.754290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-53d8f594-9388-4ff1-80f4-56129b34f89d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:10.755489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-53d8f594-9388-4ff1-80f4-56129b34f89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:10.791805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872d9d5-7e83-407c-b1f3-8cc4bdd8297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:10.792454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872d9d5-7e83-407c-b1f3-8cc4bdd8297a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.792747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872d9d5-7e83-407c-b1f3-8cc4bdd8297a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.793020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872d9d5-7e83-407c-b1f3-8cc4bdd8297a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.804428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:10.805569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872d9d5-7e83-407c-b1f3-8cc4bdd8297a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:10.807041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872d9d5-7e83-407c-b1f3-8cc4bdd8297a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-3872d9d5-7e83-407c-b1f3-8cc4bdd8297a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:10.822167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:10.828876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:10.830504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:10.830504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:10.830504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:10.830504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:10.830504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 395/811] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:10.830504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872d9d5-7e83-407c-b1f3-8cc4bdd8297a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:10.838797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:10.849401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 417/812] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:10.849401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8f594-9388-4ff1-80f4-56129b34f89d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:10.938138 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872d9d5-7e83-407c-b1f3-8cc4bdd8297a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:10.964539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8f594-9388-4ff1-80f4-56129b34f89d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:10.977170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8aa39-bba7-47ce-90b3-b34b69afcdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:10.978027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8aa39-bba7-47ce-90b3-b34b69afcdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:10.978541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8aa39-bba7-47ce-90b3-b34b69afcdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:10.979100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8aa39-bba7-47ce-90b3-b34b69afcdf7 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:10.979794 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8aa39-bba7-47ce-90b3-b34b69afcdf7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:10.980106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8aa39-bba7-47ce-90b3-b34b69afcdf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:10.981645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8aa39-bba7-47ce-90b3-b34b69afcdf7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:10.992089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 396/813] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 28 06:10:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:10.992089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8aa39-bba7-47ce-90b3-b34b69afcdf7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:11.006372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bd582-0036-464f-8315-5e8d4b0f2ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:11.007036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bd582-0036-464f-8315-5e8d4b0f2ffc None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:11.007325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bd582-0036-464f-8315-5e8d4b0f2ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:11.007604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bd582-0036-464f-8315-5e8d4b0f2ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:11.018292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bd582-0036-464f-8315-5e8d4b0f2ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:11.019754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bd582-0036-464f-8315-5e8d4b0f2ffc None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-009bd582-0036-464f-8315-5e8d4b0f2ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:11.036375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:11.037395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:11.043796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:11.043796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:11.043796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:11.043796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:11.043796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 418/814] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:11.043796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bd582-0036-464f-8315-5e8d4b0f2ffc None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:11.072016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:11.089063 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:11.115830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8aa39-bba7-47ce-90b3-b34b69afcdf7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:11.149788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bd582-0036-464f-8315-5e8d4b0f2ffc None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:11.153112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff87b63-7def-46cf-88bf-f995f4452498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:11.153786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87b63-7def-46cf-88bf-f995f4452498 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:11.154073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87b63-7def-46cf-88bf-f995f4452498 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:11.154355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87b63-7def-46cf-88bf-f995f4452498 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:11.155053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff87b63-7def-46cf-88bf-f995f4452498 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:11.155363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff87b63-7def-46cf-88bf-f995f4452498 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:11.156747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff87b63-7def-46cf-88bf-f995f4452498 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:11.170323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 397/815] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:11.170973 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff87b63-7def-46cf-88bf-f995f4452498 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:11.192838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15271b2e-103d-4d01-8701-aba5a584da30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:11.193524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15271b2e-103d-4d01-8701-aba5a584da30 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:11.193817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15271b2e-103d-4d01-8701-aba5a584da30 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:11.194103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15271b2e-103d-4d01-8701-aba5a584da30 None admin] PATH_INFO: `/v3/services/network` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:11.194743 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15271b2e-103d-4d01-8701-aba5a584da30 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:11.196260 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15271b2e-103d-4d01-8701-aba5a584da30 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-15271b2e-103d-4d01-8701-aba5a584da30 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:11.206089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:11.207163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:10:11.208000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:11.211958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 419/816] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:11 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:11.212768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15271b2e-103d-4d01-8701-aba5a584da30 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:11.281720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff87b63-7def-46cf-88bf-f995f4452498 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:11.310619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:11.319441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15271b2e-103d-4d01-8701-aba5a584da30 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:11.320123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cc497-7e82-4afa-b5e4-32145c6e6952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:11.320756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cc497-7e82-4afa-b5e4-32145c6e6952 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:11.321047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cc497-7e82-4afa-b5e4-32145c6e6952 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:11.321350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cc497-7e82-4afa-b5e4-32145c6e6952 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:11.322078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cc497-7e82-4afa-b5e4-32145c6e6952 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:11.323466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cc497-7e82-4afa-b5e4-32145c6e6952 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:11.328625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:11.357349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3ddc9-03c2-45f0-b8a2-660eb2bfbfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:11.357910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3ddc9-03c2-45f0-b8a2-660eb2bfbfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:11.358205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3ddc9-03c2-45f0-b8a2-660eb2bfbfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:11.358477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3ddc9-03c2-45f0-b8a2-660eb2bfbfd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:11.359178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3ddc9-03c2-45f0-b8a2-660eb2bfbfd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:11.359505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3ddc9-03c2-45f0-b8a2-660eb2bfbfd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:11.361590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3ddc9-03c2-45f0-b8a2-660eb2bfbfd5 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:11.375092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 420/817] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:11.375937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3ddc9-03c2-45f0-b8a2-660eb2bfbfd5 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:11.456865 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 398/818] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:10:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:11.458711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cc497-7e82-4afa-b5e4-32145c6e6952 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:11.487143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3ddc9-03c2-45f0-b8a2-660eb2bfbfd5 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:11.541877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9258a-c014-431f-908d-a0ca7f511c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:11.543099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9258a-c014-431f-908d-a0ca7f511c4e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:11.543603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9258a-c014-431f-908d-a0ca7f511c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:11.544085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9258a-c014-431f-908d-a0ca7f511c4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:11.554983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9258a-c014-431f-908d-a0ca7f511c4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:11.556500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9258a-c014-431f-908d-a0ca7f511c4e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-c4d9258a-c014-431f-908d-a0ca7f511c4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:11.570751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:11.571714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:11.572614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:11.572614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:11.572614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:11.572614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:11.572614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:11.572614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:11.572614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:11.572614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:11.577386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 421/819] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:11.578394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9258a-c014-431f-908d-a0ca7f511c4e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:11.592273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:11.631686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cc497-7e82-4afa-b5e4-32145c6e6952 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:11.677904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9df528-7932-4c7e-82b7-ee647b6238b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:11.679025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df528-7932-4c7e-82b7-ee647b6238b0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:11.679517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df528-7932-4c7e-82b7-ee647b6238b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:11.679997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df528-7932-4c7e-82b7-ee647b6238b0 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:11.680906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9df528-7932-4c7e-82b7-ee647b6238b0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:11.681456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9df528-7932-4c7e-82b7-ee647b6238b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:11.683046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9df528-7932-4c7e-82b7-ee647b6238b0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:11.690543 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 399/820] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:11.692472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9df528-7932-4c7e-82b7-ee647b6238b0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:11.714170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9258a-c014-431f-908d-a0ca7f511c4e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:11.759607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:11.767210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ade6e-847b-4ac8-81ae-19c8f7b5c8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:11.772579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ade6e-847b-4ac8-81ae-19c8f7b5c8aa None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:11.772579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ade6e-847b-4ac8-81ae-19c8f7b5c8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:11.772579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ade6e-847b-4ac8-81ae-19c8f7b5c8aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:11.780014 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ade6e-847b-4ac8-81ae-19c8f7b5c8aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:11.781523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ade6e-847b-4ac8-81ae-19c8f7b5c8aa None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-a06ade6e-847b-4ac8-81ae-19c8f7b5c8aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:11.794030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:11.795197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:11.796165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:11.796165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:11.796165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:11.796165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:11.796165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:11.796165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:11.796165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:11.796165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:11.796165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:10:11.796165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:11.800221 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 422/821] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 28 06:10:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:11.801131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ade6e-847b-4ac8-81ae-19c8f7b5c8aa None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:11.825555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9df528-7932-4c7e-82b7-ee647b6238b0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:11.878462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94879dd1-e5bb-4efd-aca3-4b08878705ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:11.880135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94879dd1-e5bb-4efd-aca3-4b08878705ea None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:11.880615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94879dd1-e5bb-4efd-aca3-4b08878705ea None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:11.881086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94879dd1-e5bb-4efd-aca3-4b08878705ea None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:11.882752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94879dd1-e5bb-4efd-aca3-4b08878705ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:11.883526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94879dd1-e5bb-4efd-aca3-4b08878705ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:11.885701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94879dd1-e5bb-4efd-aca3-4b08878705ea None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:11.894626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:11.903045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 400/822] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:11.903907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94879dd1-e5bb-4efd-aca3-4b08878705ea None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:11.944033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ade6e-847b-4ac8-81ae-19c8f7b5c8aa None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:11.984773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254436a5-31a6-4dfd-a26f-2e59d9c5576e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:11.986073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254436a5-31a6-4dfd-a26f-2e59d9c5576e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:11.986319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254436a5-31a6-4dfd-a26f-2e59d9c5576e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:11.986557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254436a5-31a6-4dfd-a26f-2e59d9c5576e None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:11.987254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254436a5-31a6-4dfd-a26f-2e59d9c5576e None admin] RBAC: Adding query filter params (type=network) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:11.987572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254436a5-31a6-4dfd-a26f-2e59d9c5576e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:11.988957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254436a5-31a6-4dfd-a26f-2e59d9c5576e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:11.997730 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 423/823] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:11.997730 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254436a5-31a6-4dfd-a26f-2e59d9c5576e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:12.017002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94879dd1-e5bb-4efd-aca3-4b08878705ea None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:12.033920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:12.056122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71987d9f-d6a0-4649-b1c5-f5674cec8581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:12.056738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71987d9f-d6a0-4649-b1c5-f5674cec8581 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:12.057022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71987d9f-d6a0-4649-b1c5-f5674cec8581 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:12.057298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71987d9f-d6a0-4649-b1c5-f5674cec8581 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:12.058118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71987d9f-d6a0-4649-b1c5-f5674cec8581 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:12.058537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71987d9f-d6a0-4649-b1c5-f5674cec8581 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:12.060080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71987d9f-d6a0-4649-b1c5-f5674cec8581 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:12.084039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 401/824] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 28 06:10:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:12.084039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7a899-2267-4ab4-964f-91662cdf30b3 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:12.084039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7a899-2267-4ab4-964f-91662cdf30b3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:12.084039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7a899-2267-4ab4-964f-91662cdf30b3 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:12.084039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 402/825] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:12.084482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7a899-2267-4ab4-964f-91662cdf30b3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:12.111574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254436a5-31a6-4dfd-a26f-2e59d9c5576e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:12.143865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:12.151284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991c47-3991-4192-bd0d-86730ff24167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:12.155991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991c47-3991-4192-bd0d-86730ff24167 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:12.155991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991c47-3991-4192-bd0d-86730ff24167 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:12.155991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991c47-3991-4192-bd0d-86730ff24167 None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/39c6f0ee9b78413eb8c5efc7adf145b6/roles/31125b687d5e4a29b891ade41748f51e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:12.176878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3991c47-3991-4192-bd0d-86730ff24167 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=39c6f0ee9b78413eb8c5efc7adf145b6, role_id=31125b687d5e4a29b891ade41748f51e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:12.179262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3991c47-3991-4192-bd0d-86730ff24167 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:12.206371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7a899-2267-4ab4-964f-91662cdf30b3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:12.241448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1086a-ec6c-45d3-a818-74d3eb79f735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:12.242059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1086a-ec6c-45d3-a818-74d3eb79f735 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:12.242342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1086a-ec6c-45d3-a818-74d3eb79f735 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:12.242618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1086a-ec6c-45d3-a818-74d3eb79f735 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:12.243327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1086a-ec6c-45d3-a818-74d3eb79f735 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:12.244788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1086a-ec6c-45d3-a818-74d3eb79f735 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:12.257861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 424/826] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:10:11 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/39c6f0ee9b78413eb8c5efc7adf145b6/roles/31125b687d5e4a29b891ade41748f51e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:10:12.257861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991c47-3991-4192-bd0d-86730ff24167 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:12.354004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 403/827] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:10:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:12.355972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ab537-ec61-4bbc-aca8-43f297a8bcdd None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:12.356547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ab537-ec61-4bbc-aca8-43f297a8bcdd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:12.356837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ab537-ec61-4bbc-aca8-43f297a8bcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:12.383958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991c47-3991-4192-bd0d-86730ff24167 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:12.388689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d21ab537-ec61-4bbc-aca8-43f297a8bcdd None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:12.390245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d21ab537-ec61-4bbc-aca8-43f297a8bcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:12.444842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:12.472947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ff671-fdce-4eee-9254-02f6eca6593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:12.475839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ff671-fdce-4eee-9254-02f6eca6593b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:12.476322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ff671-fdce-4eee-9254-02f6eca6593b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:12.476754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ff671-fdce-4eee-9254-02f6eca6593b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:12.513195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 404/828] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:12.537175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ab537-ec61-4bbc-aca8-43f297a8bcdd None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:12.563846 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:12.596478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ff671-fdce-4eee-9254-02f6eca6593b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:12.598838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ff671-fdce-4eee-9254-02f6eca6593b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:12.689636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ab537-ec61-4bbc-aca8-43f297a8bcdd None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:12.741388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 425/829] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:10:12.745716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ff671-fdce-4eee-9254-02f6eca6593b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:12.757182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40094b-cbe8-4dd9-989f-79d63dbccabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:12.758256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40094b-cbe8-4dd9-989f-79d63dbccabf None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:12.758829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40094b-cbe8-4dd9-989f-79d63dbccabf None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:12.759379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40094b-cbe8-4dd9-989f-79d63dbccabf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:12.774673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40094b-cbe8-4dd9-989f-79d63dbccabf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4c40094b-cbe8-4dd9-989f-79d63dbccabf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:12.780372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:12.781565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:12.783295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:12.788235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 405/830] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:12.805009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40094b-cbe8-4dd9-989f-79d63dbccabf None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:12.830634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:12.838796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:12.863649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ff671-fdce-4eee-9254-02f6eca6593b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:12.903326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7e963-165b-484c-8155-1edb25a19803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:12.903982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e963-165b-484c-8155-1edb25a19803 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:12.904272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e963-165b-484c-8155-1edb25a19803 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:12.904556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e963-165b-484c-8155-1edb25a19803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:12.917099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7e963-165b-484c-8155-1edb25a19803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:12.918406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7e963-165b-484c-8155-1edb25a19803 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:12.924367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40094b-cbe8-4dd9-989f-79d63dbccabf None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-14f7e963-165b-484c-8155-1edb25a19803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:12.935936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:12.936861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:12.937898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:12.937898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:12.937898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:12.937898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:12.941821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 426/831] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:12.943391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7e963-165b-484c-8155-1edb25a19803 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:12.964503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbee28-4499-46a4-b3cd-49d3cb5cd3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:12.965451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbee28-4499-46a4-b3cd-49d3cb5cd3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:12.965958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbee28-4499-46a4-b3cd-49d3cb5cd3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:12.966436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbee28-4499-46a4-b3cd-49d3cb5cd3e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:12.967424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbee28-4499-46a4-b3cd-49d3cb5cd3e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:12.967935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbee28-4499-46a4-b3cd-49d3cb5cd3e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:12.970195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbee28-4499-46a4-b3cd-49d3cb5cd3e6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:12.979897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 406/832] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:10:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:12.986820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbee28-4499-46a4-b3cd-49d3cb5cd3e6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:13.067807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7e963-165b-484c-8155-1edb25a19803 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:13.098750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:13.107291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:13.117853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58d79e-2b84-4b44-b804-7906473010ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:13.118821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58d79e-2b84-4b44-b804-7906473010ae None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.119312 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58d79e-2b84-4b44-b804-7906473010ae None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.121010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58d79e-2b84-4b44-b804-7906473010ae None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:13.122005 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58d79e-2b84-4b44-b804-7906473010ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:13.122499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58d79e-2b84-4b44-b804-7906473010ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:13.124164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58d79e-2b84-4b44-b804-7906473010ae None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:13.128711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbee28-4499-46a4-b3cd-49d3cb5cd3e6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:13.143271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 427/833] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:13.145001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58d79e-2b84-4b44-b804-7906473010ae None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:13.166880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b85a3-02da-4dcd-afe6-7c02a5843800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:13.167678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b85a3-02da-4dcd-afe6-7c02a5843800 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.168039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b85a3-02da-4dcd-afe6-7c02a5843800 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.168414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b85a3-02da-4dcd-afe6-7c02a5843800 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:13.180602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b85a3-02da-4dcd-afe6-7c02a5843800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:13.182258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b85a3-02da-4dcd-afe6-7c02a5843800 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-be6b85a3-02da-4dcd-afe6-7c02a5843800 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:13.197873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:13.198851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:13.200046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:13.200046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:13.200046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:13.200046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:13.204580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 407/834] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:13.205824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b85a3-02da-4dcd-afe6-7c02a5843800 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:13.262185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58d79e-2b84-4b44-b804-7906473010ae None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:13.316841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd423f-b3f2-4cf2-b37b-a37e39c2eb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:13.318176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd423f-b3f2-4cf2-b37b-a37e39c2eb93 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.318464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd423f-b3f2-4cf2-b37b-a37e39c2eb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.318831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd423f-b3f2-4cf2-b37b-a37e39c2eb93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:13.328419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd423f-b3f2-4cf2-b37b-a37e39c2eb93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:13.330965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd423f-b3f2-4cf2-b37b-a37e39c2eb93 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-fddd423f-b3f2-4cf2-b37b-a37e39c2eb93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:13.347388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:13.349023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:13.351036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:13.351036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:13.351036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:13.351036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:13.351036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:13.351036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:13.351036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:13.351036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:13.353508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 428/835] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:13.354594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd423f-b3f2-4cf2-b37b-a37e39c2eb93 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:13.356867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b85a3-02da-4dcd-afe6-7c02a5843800 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:13.399449 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:13.406111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c5dbf-76ac-4170-96ef-b3e06f5b2586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:13.407121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c5dbf-76ac-4170-96ef-b3e06f5b2586 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.407681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c5dbf-76ac-4170-96ef-b3e06f5b2586 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.408184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c5dbf-76ac-4170-96ef-b3e06f5b2586 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:13.409120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c5dbf-76ac-4170-96ef-b3e06f5b2586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:13.410229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c5dbf-76ac-4170-96ef-b3e06f5b2586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:13.412336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c5dbf-76ac-4170-96ef-b3e06f5b2586 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:13.421505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:13.429259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 408/836] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:13.435767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c5dbf-76ac-4170-96ef-b3e06f5b2586 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:13.495820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd423f-b3f2-4cf2-b37b-a37e39c2eb93 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:13.535065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd502dca-9839-402e-a52b-9d2251ec3634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:13.536095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502dca-9839-402e-a52b-9d2251ec3634 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.536691 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502dca-9839-402e-a52b-9d2251ec3634 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.537177 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502dca-9839-402e-a52b-9d2251ec3634 None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:13.538283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd502dca-9839-402e-a52b-9d2251ec3634 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:13.538841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd502dca-9839-402e-a52b-9d2251ec3634 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:13.540661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd502dca-9839-402e-a52b-9d2251ec3634 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:13.553845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c5dbf-76ac-4170-96ef-b3e06f5b2586 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:13.556296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 429/837] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:13.561873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd502dca-9839-402e-a52b-9d2251ec3634 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:13.604457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92df98-e46a-43a0-b52e-34b167f9afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:13.605717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92df98-e46a-43a0-b52e-34b167f9afd1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.606567 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92df98-e46a-43a0-b52e-34b167f9afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.607297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92df98-e46a-43a0-b52e-34b167f9afd1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:13.616967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92df98-e46a-43a0-b52e-34b167f9afd1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:13.619210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92df98-e46a-43a0-b52e-34b167f9afd1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ca92df98-e46a-43a0-b52e-34b167f9afd1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:13.635651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:13.636910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:13.638518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:13.638518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:13.638518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:13.638518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:13.638518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:13.638518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:13.638518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:13.638518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:13.638518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:10:13.638518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:13.639438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 409/838] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:13.644749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92df98-e46a-43a0-b52e-34b167f9afd1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:13.696439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd502dca-9839-402e-a52b-9d2251ec3634 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:13.700832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:13.710199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:13.733775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7be554-c795-4512-8dc4-ee2bbdeda7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:13.734390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7be554-c795-4512-8dc4-ee2bbdeda7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.735070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7be554-c795-4512-8dc4-ee2bbdeda7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.735070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7be554-c795-4512-8dc4-ee2bbdeda7c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:13.736015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7be554-c795-4512-8dc4-ee2bbdeda7c4 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=5706aa2d6bf54f05badf9588f51f4677) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:13.736443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7be554-c795-4512-8dc4-ee2bbdeda7c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:13.737911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7be554-c795-4512-8dc4-ee2bbdeda7c4 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:13.749179 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92df98-e46a-43a0-b52e-34b167f9afd1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:13.753499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 430/839] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:13 2022] GET /identity/v3/role_assignments?user.id=5706aa2d6bf54f05badf9588f51f4677&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:13.755463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afe928-30c0-44f8-9cc5-bfc75e17a5eb None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.755985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afe928-30c0-44f8-9cc5-bfc75e17a5eb None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.756474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afe928-30c0-44f8-9cc5-bfc75e17a5eb None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:13.759455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 431/840] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:13.765822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c42e0-40bb-4459-b8aa-84fd6ade806e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.767892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c42e0-40bb-4459-b8aa-84fd6ade806e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.768471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c42e0-40bb-4459-b8aa-84fd6ade806e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:13.795877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0914c99-cadb-4f0a-978d-b388cb98036c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:13.796530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0914c99-cadb-4f0a-978d-b388cb98036c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.796854 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0914c99-cadb-4f0a-978d-b388cb98036c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.797127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0914c99-cadb-4f0a-978d-b388cb98036c None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:13.798236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-bb4c42e0-40bb-4459-b8aa-84fd6ade806e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:13.798611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0914c99-cadb-4f0a-978d-b388cb98036c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:13.798926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0914c99-cadb-4f0a-978d-b388cb98036c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:13.800198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-bb4c42e0-40bb-4459-b8aa-84fd6ade806e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:13.800474 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0914c99-cadb-4f0a-978d-b388cb98036c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:13.826807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 410/841] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 28 06:10:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:13.827995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0914c99-cadb-4f0a-978d-b388cb98036c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:13.899914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 432/842] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:13.927559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c42e0-40bb-4459-b8aa-84fd6ade806e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:13.956241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0914c99-cadb-4f0a-978d-b388cb98036c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:13.967880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:13.997051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a3179-cdd0-4d37-8046-2c54c3737cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:13.997982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a3179-cdd0-4d37-8046-2c54c3737cdc None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:13.998370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a3179-cdd0-4d37-8046-2c54c3737cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:13.998953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a3179-cdd0-4d37-8046-2c54c3737cdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:14.058705 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c42e0-40bb-4459-b8aa-84fd6ade806e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:14.061923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:14.081996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a3179-cdd0-4d37-8046-2c54c3737cdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:14.083662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a3179-cdd0-4d37-8046-2c54c3737cdc None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:14.112025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23f7d5-e4da-42b8-95d9-b0d989ccce6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:14.113621 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23f7d5-e4da-42b8-95d9-b0d989ccce6b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:14.114430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23f7d5-e4da-42b8-95d9-b0d989ccce6b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:14.115107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23f7d5-e4da-42b8-95d9-b0d989ccce6b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:14.115900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23f7d5-e4da-42b8-95d9-b0d989ccce6b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:14.118378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23f7d5-e4da-42b8-95d9-b0d989ccce6b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-bc23f7d5-e4da-42b8-95d9-b0d989ccce6b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:14.131544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:14.132792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:10:14.133869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:14.137430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 433/843] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:10:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:14.143545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23f7d5-e4da-42b8-95d9-b0d989ccce6b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:14.221909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 411/844] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:10:14.223507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a3179-cdd0-4d37-8046-2c54c3737cdc None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:14.236609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:14.251833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23f7d5-e4da-42b8-95d9-b0d989ccce6b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:14.289610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13197a55-aef7-41b6-ae55-eda6ad8a0cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:14.290292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197a55-aef7-41b6-ae55-eda6ad8a0cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:14.290581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197a55-aef7-41b6-ae55-eda6ad8a0cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:14.290861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197a55-aef7-41b6-ae55-eda6ad8a0cc0 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:14.291652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13197a55-aef7-41b6-ae55-eda6ad8a0cc0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:14.291901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13197a55-aef7-41b6-ae55-eda6ad8a0cc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:14.293492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13197a55-aef7-41b6-ae55-eda6ad8a0cc0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:14.299452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 434/845] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:10:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:14.304735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13197a55-aef7-41b6-ae55-eda6ad8a0cc0 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:14.338550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:14.358546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a3179-cdd0-4d37-8046-2c54c3737cdc None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:14.404285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5132c91-6e8e-4e12-8fce-d0ce456161b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:14.404944 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5132c91-6e8e-4e12-8fce-d0ce456161b3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:14.405220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5132c91-6e8e-4e12-8fce-d0ce456161b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:14.405524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5132c91-6e8e-4e12-8fce-d0ce456161b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:14.416116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13197a55-aef7-41b6-ae55-eda6ad8a0cc0 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:14.417395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5132c91-6e8e-4e12-8fce-d0ce456161b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:14.419128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5132c91-6e8e-4e12-8fce-d0ce456161b3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-d5132c91-6e8e-4e12-8fce-d0ce456161b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:14.435780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:14.436726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:14.437621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:14.437621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:14.437621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:14.437621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:14.441835 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 412/846] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:14.444303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5132c91-6e8e-4e12-8fce-d0ce456161b3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:14.453239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909f115-9513-4a2d-8cd2-ac1b7a5a9807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:14.454010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909f115-9513-4a2d-8cd2-ac1b7a5a9807 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:14.454295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909f115-9513-4a2d-8cd2-ac1b7a5a9807 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:14.454578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909f115-9513-4a2d-8cd2-ac1b7a5a9807 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:14.455277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909f115-9513-4a2d-8cd2-ac1b7a5a9807 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:14.455687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909f115-9513-4a2d-8cd2-ac1b7a5a9807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:14.457153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909f115-9513-4a2d-8cd2-ac1b7a5a9807 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:14.462721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 435/847] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:10:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:14.507259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f42aae-53b2-416a-88ec-33b698f49caa None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:14.507605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f42aae-53b2-416a-88ec-33b698f49caa None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:14.507837 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f42aae-53b2-416a-88ec-33b698f49caa None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:14.513633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 436/848] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:10:14.515289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac929ce-d543-48f6-9734-badb41b83018 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:14.515578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac929ce-d543-48f6-9734-badb41b83018 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:14.515859 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac929ce-d543-48f6-9734-badb41b83018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:14.536338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-9ac929ce-d543-48f6-9734-badb41b83018 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:14.537753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9ac929ce-d543-48f6-9734-badb41b83018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:14.538348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:14.599624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5132c91-6e8e-4e12-8fce-d0ce456161b3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:14.644712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 437/849] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:14.654717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b02b7-e090-4ab4-9474-67b2cccfad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:14.655346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b02b7-e090-4ab4-9474-67b2cccfad72 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:14.655637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b02b7-e090-4ab4-9474-67b2cccfad72 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:14.655921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b02b7-e090-4ab4-9474-67b2cccfad72 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:14.656628 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b02b7-e090-4ab4-9474-67b2cccfad72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:14.656943 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b02b7-e090-4ab4-9474-67b2cccfad72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:14.658494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b02b7-e090-4ab4-9474-67b2cccfad72 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:14.666132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac929ce-d543-48f6-9734-badb41b83018 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:14.678188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 413/850] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:14.683664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b02b7-e090-4ab4-9474-67b2cccfad72 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:14.702232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:14.794329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b02b7-e090-4ab4-9474-67b2cccfad72 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:14.805541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac929ce-d543-48f6-9734-badb41b83018 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:14.829892 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:14.843666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738abcce-4bc7-405a-b53d-fea4d6feefa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:14.844249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479314c4-5676-4a2d-a096-b9cbb2f5e7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:14.844604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738abcce-4bc7-405a-b53d-fea4d6feefa3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:14.845524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738abcce-4bc7-405a-b53d-fea4d6feefa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:14.845863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738abcce-4bc7-405a-b53d-fea4d6feefa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:14.846264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479314c4-5676-4a2d-a096-b9cbb2f5e7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:14.846776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479314c4-5676-4a2d-a096-b9cbb2f5e7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:14.847130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479314c4-5676-4a2d-a096-b9cbb2f5e7f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:14.855336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738abcce-4bc7-405a-b53d-fea4d6feefa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:14.857269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738abcce-4bc7-405a-b53d-fea4d6feefa3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:14.860418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479314c4-5676-4a2d-a096-b9cbb2f5e7f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-479314c4-5676-4a2d-a096-b9cbb2f5e7f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:14.866001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:14.867776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:14.869498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 438/851] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:10:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-738abcce-4bc7-405a-b53d-fea4d6feefa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:14.874786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:14.875852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:14.876980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:14.876980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:14.876980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:14.876980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:14.876980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:14.876980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:14.876980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:14.876980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:14.877546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479314c4-5676-4a2d-a096-b9cbb2f5e7f2 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:14.879191 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 414/852] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:14.884581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738abcce-4bc7-405a-b53d-fea4d6feefa3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:14.979297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:14.996079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479314c4-5676-4a2d-a096-b9cbb2f5e7f2 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:14.996278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738abcce-4bc7-405a-b53d-fea4d6feefa3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:15.033041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336f728-396a-48a9-bbbf-c826212a8586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:15.034155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336f728-396a-48a9-bbbf-c826212a8586 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:15.034598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336f728-396a-48a9-bbbf-c826212a8586 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:15.035052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336f728-396a-48a9-bbbf-c826212a8586 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:15.035365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24386b-f3b4-4afe-b54c-ca8f540f48a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:15.036105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24386b-f3b4-4afe-b54c-ca8f540f48a5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:15.036263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336f728-396a-48a9-bbbf-c826212a8586 None admin] RBAC: Adding query filter params (name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:15.036578 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24386b-f3b4-4afe-b54c-ca8f540f48a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:15.036867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336f728-396a-48a9-bbbf-c826212a8586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:15.037020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24386b-f3b4-4afe-b54c-ca8f540f48a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:15.037968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24386b-f3b4-4afe-b54c-ca8f540f48a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:15.038379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24386b-f3b4-4afe-b54c-ca8f540f48a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:15.038920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336f728-396a-48a9-bbbf-c826212a8586 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:15.040084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24386b-f3b4-4afe-b54c-ca8f540f48a5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:15.053465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 439/853] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:10:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:15.054653 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336f728-396a-48a9-bbbf-c826212a8586 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:15.061204 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 415/854] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:15.062265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24386b-f3b4-4afe-b54c-ca8f540f48a5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:15.113678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:15.161588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336f728-396a-48a9-bbbf-c826212a8586 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:15.175857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24386b-f3b4-4afe-b54c-ca8f540f48a5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:15.206255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f9c7c-1f80-45df-b460-344cdd6f5882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:15.207250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f9c7c-1f80-45df-b460-344cdd6f5882 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:15.207726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f9c7c-1f80-45df-b460-344cdd6f5882 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:15.208191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f9c7c-1f80-45df-b460-344cdd6f5882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:15.215479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fc773-f05b-4f43-b1f4-a2816c716305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:15.216097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fc773-f05b-4f43-b1f4-a2816c716305 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:15.216378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fc773-f05b-4f43-b1f4-a2816c716305 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:15.216654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fc773-f05b-4f43-b1f4-a2816c716305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:15.217637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fc773-f05b-4f43-b1f4-a2816c716305 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=77571de30e934d6590ff7b58a000f5e2) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:15.217893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fc773-f05b-4f43-b1f4-a2816c716305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:15.219332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fc773-f05b-4f43-b1f4-a2816c716305 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:15.222646 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f9c7c-1f80-45df-b460-344cdd6f5882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:15.224357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f9c7c-1f80-45df-b460-344cdd6f5882 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:15.233859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 416/855] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:15 2022] GET /identity/v3/role_assignments?user.id=77571de30e934d6590ff7b58a000f5e2&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:15.237663 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-586f9c7c-1f80-45df-b460-344cdd6f5882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:15.246088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:15.247250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:15.248369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:15.248369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:15.248369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:15.248369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:15.250008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 440/856] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:15.256665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fc773-f05b-4f43-b1f4-a2816c716305 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:15.389491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fc773-f05b-4f43-b1f4-a2816c716305 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:15.398955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:15.431119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd5f31-b193-4e57-a6b5-9fbd253d547a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:15.432152 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd5f31-b193-4e57-a6b5-9fbd253d547a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:15.432717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd5f31-b193-4e57-a6b5-9fbd253d547a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:15.433227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd5f31-b193-4e57-a6b5-9fbd253d547a None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:15.434199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd5f31-b193-4e57-a6b5-9fbd253d547a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:15.434717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd5f31-b193-4e57-a6b5-9fbd253d547a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:15.436334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd5f31-b193-4e57-a6b5-9fbd253d547a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:15.450265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 417/857] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:15.456112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f9c7c-1f80-45df-b460-344cdd6f5882 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:15.569594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f9c7c-1f80-45df-b460-344cdd6f5882 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:15.604699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f7cd9-4087-4b95-aebb-7419ae925ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:15.605369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f7cd9-4087-4b95-aebb-7419ae925ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:15.605655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f7cd9-4087-4b95-aebb-7419ae925ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:15.605934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f7cd9-4087-4b95-aebb-7419ae925ca0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:15.614770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f7cd9-4087-4b95-aebb-7419ae925ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:15.616225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f7cd9-4087-4b95-aebb-7419ae925ca0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-b20f7cd9-4087-4b95-aebb-7419ae925ca0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:15.628936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:15.629946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:15.630888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 441/858] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 28 06:10:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:15.638173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd5f31-b193-4e57-a6b5-9fbd253d547a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:15.746714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd5f31-b193-4e57-a6b5-9fbd253d547a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:15.783746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd5cae-9e4e-4f34-804e-e497f53ca984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:15.784387 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5cae-9e4e-4f34-804e-e497f53ca984 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:15.784676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5cae-9e4e-4f34-804e-e497f53ca984 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:15.784955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5cae-9e4e-4f34-804e-e497f53ca984 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:15.785797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd5cae-9e4e-4f34-804e-e497f53ca984 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:15.786108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd5cae-9e4e-4f34-804e-e497f53ca984 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:15.787558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd5cae-9e4e-4f34-804e-e497f53ca984 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:15.806609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 418/859] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 28 06:10:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:15.812224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f7cd9-4087-4b95-aebb-7419ae925ca0 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:15.879732 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86e051-1d89-4b3e-acf4-eec1e8be7ff8 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:15.880031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86e051-1d89-4b3e-acf4-eec1e8be7ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:15.880222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86e051-1d89-4b3e-acf4-eec1e8be7ff8 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:15.880821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:15.881736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 419/860] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:10:15.886855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f680f-1a19-4bb3-a31a-65dcec044621 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:15.887141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f680f-1a19-4bb3-a31a-65dcec044621 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:15.887421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f680f-1a19-4bb3-a31a-65dcec044621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:15.912021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e61f680f-1a19-4bb3-a31a-65dcec044621 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:15.913493 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e61f680f-1a19-4bb3-a31a-65dcec044621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:15.932768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:15.940960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f7cd9-4087-4b95-aebb-7419ae925ca0 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:15.986597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a1e63-447b-4cf9-b223-c2516285d0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:15.987500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a1e63-447b-4cf9-b223-c2516285d0cd None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:15.987922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a1e63-447b-4cf9-b223-c2516285d0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:15.988338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a1e63-447b-4cf9-b223-c2516285d0cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:15.993759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 420/861] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:16.007317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f680f-1a19-4bb3-a31a-65dcec044621 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:16.066566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a1e63-447b-4cf9-b223-c2516285d0cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:16.068415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a1e63-447b-4cf9-b223-c2516285d0cd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:16.114750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f680f-1a19-4bb3-a31a-65dcec044621 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:16.133994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:16.151566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778321c2-9321-421a-a661-e580af4e8bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:16.152504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778321c2-9321-421a-a661-e580af4e8bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:16.152974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778321c2-9321-421a-a661-e580af4e8bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:16.153432 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778321c2-9321-421a-a661-e580af4e8bc7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:16.154227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778321c2-9321-421a-a661-e580af4e8bc7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:16.155818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778321c2-9321-421a-a661-e580af4e8bc7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-778321c2-9321-421a-a661-e580af4e8bc7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:16.166005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:16.166984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:10:16.167836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:16.170919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 421/862] 10.222.0.69 () {56 vars in 1132 bytes} [Thu Jul 28 06:10:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:16.177298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778321c2-9321-421a-a661-e580af4e8bc7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:16.197415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 442/863] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:10:16.199666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a1e63-447b-4cf9-b223-c2516285d0cd None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:16.205889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:16.311821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a1e63-447b-4cf9-b223-c2516285d0cd None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:16.314954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778321c2-9321-421a-a661-e580af4e8bc7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:16.356065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a01c8-f1e4-469b-bfb9-ba90632c93d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:16.356696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a01c8-f1e4-469b-bfb9-ba90632c93d9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:16.356983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a01c8-f1e4-469b-bfb9-ba90632c93d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:16.357461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a01c8-f1e4-469b-bfb9-ba90632c93d9 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:16.358166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a01c8-f1e4-469b-bfb9-ba90632c93d9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:16.358480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a01c8-f1e4-469b-bfb9-ba90632c93d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:16.360595 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a01c8-f1e4-469b-bfb9-ba90632c93d9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:16.366067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b210e-a108-42f9-96ff-b06743c680c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:16.366956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b210e-a108-42f9-96ff-b06743c680c1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:16.367467 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b210e-a108-42f9-96ff-b06743c680c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:16.367954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b210e-a108-42f9-96ff-b06743c680c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:16.372671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 422/864] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:16.373609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a01c8-f1e4-469b-bfb9-ba90632c93d9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:16.381631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b210e-a108-42f9-96ff-b06743c680c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:16.384078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b210e-a108-42f9-96ff-b06743c680c1 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-389b210e-a108-42f9-96ff-b06743c680c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:16.405787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:16.406742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:16.408502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:16.408502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:16.408502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:16.408502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:16.408502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 443/865] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:16.413883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b210e-a108-42f9-96ff-b06743c680c1 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:16.434715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:16.473760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:16.501903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a01c8-f1e4-469b-bfb9-ba90632c93d9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:16.552228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b210e-a108-42f9-96ff-b06743c680c1 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:16.553479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad100fad-b0c0-4252-9a28-8e05872dce29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:16.554598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad100fad-b0c0-4252-9a28-8e05872dce29 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:16.555965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad100fad-b0c0-4252-9a28-8e05872dce29 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:16.556428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad100fad-b0c0-4252-9a28-8e05872dce29 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:16.557421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad100fad-b0c0-4252-9a28-8e05872dce29 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:16.557945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad100fad-b0c0-4252-9a28-8e05872dce29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:16.559524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad100fad-b0c0-4252-9a28-8e05872dce29 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:16.567195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 423/866] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:16.606391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4c4a2-be4f-49f0-bdc2-104d7bb24cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:16.607514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4c4a2-be4f-49f0-bdc2-104d7bb24cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:16.608316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4c4a2-be4f-49f0-bdc2-104d7bb24cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:16.608818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4c4a2-be4f-49f0-bdc2-104d7bb24cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:16.609945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4c4a2-be4f-49f0-bdc2-104d7bb24cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:16.611338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4c4a2-be4f-49f0-bdc2-104d7bb24cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:16.612990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4c4a2-be4f-49f0-bdc2-104d7bb24cd1 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:16.627874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 444/867] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:16.633103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad100fad-b0c0-4252-9a28-8e05872dce29 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:16.689655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06501e69-a866-4f3f-b429-dfd6bb0b3289 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:16.690281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06501e69-a866-4f3f-b429-dfd6bb0b3289 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:16.691226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06501e69-a866-4f3f-b429-dfd6bb0b3289 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:16.693598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 445/868] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:16.703576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94aa34-5d34-4d6d-b37a-b8abf06a7064 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:16.704428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94aa34-5d34-4d6d-b37a-b8abf06a7064 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:16.705287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94aa34-5d34-4d6d-b37a-b8abf06a7064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:16.730430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-3d94aa34-5d34-4d6d-b37a-b8abf06a7064 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:16.732624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3d94aa34-5d34-4d6d-b37a-b8abf06a7064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:16.758998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad100fad-b0c0-4252-9a28-8e05872dce29 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:16.800698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c050967-38c8-4d4b-a961-f7bac762e9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:16.801871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c050967-38c8-4d4b-a961-f7bac762e9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:16.802333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c050967-38c8-4d4b-a961-f7bac762e9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:16.802787 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c050967-38c8-4d4b-a961-f7bac762e9c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:16.813520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c050967-38c8-4d4b-a961-f7bac762e9c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:16.815225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c050967-38c8-4d4b-a961-f7bac762e9c2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:16.828019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 446/869] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-1c050967-38c8-4d4b-a961-f7bac762e9c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:16.828534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:16.830315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:16.831296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:16.831296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:16.831296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:16.831296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:16.831296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:16.831296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:16.831296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:16.831296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:16.834654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 424/870] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:16.840600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94aa34-5d34-4d6d-b37a-b8abf06a7064 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:16.845641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:16.853057 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c050967-38c8-4d4b-a961-f7bac762e9c2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:16.864306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:16.974895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c050967-38c8-4d4b-a961-f7bac762e9c2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:16.980495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94aa34-5d34-4d6d-b37a-b8abf06a7064 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:17.018035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0d6d3-d047-47fd-a861-f716af7e0c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:17.019061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0d6d3-d047-47fd-a861-f716af7e0c21 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:17.019548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0d6d3-d047-47fd-a861-f716af7e0c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:17.020023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0d6d3-d047-47fd-a861-f716af7e0c21 None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:17.020973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0d6d3-d047-47fd-a861-f716af7e0c21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:17.021561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0d6d3-d047-47fd-a861-f716af7e0c21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:17.021802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ffa95-4728-4331-b225-34c0cd0faf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:17.022430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ffa95-4728-4331-b225-34c0cd0faf43 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:17.022720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ffa95-4728-4331-b225-34c0cd0faf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:17.023003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ffa95-4728-4331-b225-34c0cd0faf43 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:17.023789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ffa95-4728-4331-b225-34c0cd0faf43 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:17.024250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0d6d3-d047-47fd-a861-f716af7e0c21 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:17.025213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ffa95-4728-4331-b225-34c0cd0faf43 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:17.037590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 447/871] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:17.043617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0d6d3-d047-47fd-a861-f716af7e0c21 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:17.093818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 425/872] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:10:16 2022] POST /identity/v3/services => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:10:17.127423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:17.171699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0d6d3-d047-47fd-a861-f716af7e0c21 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:17.218296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102f666-d4d0-408b-87e0-45a109f22824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:17.218975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102f666-d4d0-408b-87e0-45a109f22824 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:17.219207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102f666-d4d0-408b-87e0-45a109f22824 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:17.220078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102f666-d4d0-408b-87e0-45a109f22824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:17.221655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102f666-d4d0-408b-87e0-45a109f22824 None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=39c6f0ee9b78413eb8c5efc7adf145b6) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:17.222107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102f666-d4d0-408b-87e0-45a109f22824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:17.223617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102f666-d4d0-408b-87e0-45a109f22824 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:17.231104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 448/873] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:17 2022] GET /identity/v3/role_assignments?user.id=39c6f0ee9b78413eb8c5efc7adf145b6&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:17.460150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f54327-9328-46d9-a49b-c909122db0da None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:17.460664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f54327-9328-46d9-a49b-c909122db0da None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:17.460740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f54327-9328-46d9-a49b-c909122db0da None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:17.462382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 426/874] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:17.469016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab19d6-938a-4c15-a664-66160f6b5e1a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:17.470295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab19d6-938a-4c15-a664-66160f6b5e1a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:17.470782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab19d6-938a-4c15-a664-66160f6b5e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:17.491169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-40ab19d6-938a-4c15-a664-66160f6b5e1a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:17.492605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-40ab19d6-938a-4c15-a664-66160f6b5e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:17.563990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 449/875] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:17.586777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f54327-9328-46d9-a49b-c909122db0da None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:17.660835 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:17.699214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f54327-9328-46d9-a49b-c909122db0da None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:17.734791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06046097-03a7-4074-825a-2056d6623b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:17.736107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06046097-03a7-4074-825a-2056d6623b29 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:17.736398 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06046097-03a7-4074-825a-2056d6623b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:17.736737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06046097-03a7-4074-825a-2056d6623b29 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:17.737410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06046097-03a7-4074-825a-2056d6623b29 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:17.738826 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06046097-03a7-4074-825a-2056d6623b29 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-06046097-03a7-4074-825a-2056d6623b29 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:17.748444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:17.749469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:17.750315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 427/876] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:17.758270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab19d6-938a-4c15-a664-66160f6b5e1a None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:17.821709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:17.871839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab19d6-938a-4c15-a664-66160f6b5e1a None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:17.908539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064f016-6760-4994-932c-fe3a4d818121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:17.909166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064f016-6760-4994-932c-fe3a4d818121 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:17.909480 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064f016-6760-4994-932c-fe3a4d818121 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:17.909759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064f016-6760-4994-932c-fe3a4d818121 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:17.910446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064f016-6760-4994-932c-fe3a4d818121 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:17.910750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064f016-6760-4994-932c-fe3a4d818121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:17.912125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064f016-6760-4994-932c-fe3a4d818121 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:17.917331 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 450/877] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 28 06:10:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:17.923159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06046097-03a7-4074-825a-2056d6623b29 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:18.036748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06046097-03a7-4074-825a-2056d6623b29 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:18.066271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:18.072991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7592c-53e4-4877-9d51-ca46ff345626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:18.073923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7592c-53e4-4877-9d51-ca46ff345626 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:18.074630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7592c-53e4-4877-9d51-ca46ff345626 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:18.075136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7592c-53e4-4877-9d51-ca46ff345626 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:18.076326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7592c-53e4-4877-9d51-ca46ff345626 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:18.076844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7592c-53e4-4877-9d51-ca46ff345626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:18.078458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7592c-53e4-4877-9d51-ca46ff345626 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:18.083816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 428/878] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 28 06:10:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:18.744768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c9731-2d09-49a7-a78d-8d880e644acd None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:18.745348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c9731-2d09-49a7-a78d-8d880e644acd None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:18.745348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c9731-2d09-49a7-a78d-8d880e644acd None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:18.746941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 451/879] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:18.752744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb398d-4adb-4918-9558-356392d37130 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:18.753215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb398d-4adb-4918-9558-356392d37130 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:18.753321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb398d-4adb-4918-9558-356392d37130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:18.773238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-dedb398d-4adb-4918-9558-356392d37130 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:18.774447 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dedb398d-4adb-4918-9558-356392d37130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:18.846170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 429/880] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:18.868101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c9731-2d09-49a7-a78d-8d880e644acd None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:18.978723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c9731-2d09-49a7-a78d-8d880e644acd None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:18.996574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:19.017171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b42b8e-9796-4297-88e7-de2ea3c296bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:19.017833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b42b8e-9796-4297-88e7-de2ea3c296bc None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:19.018122 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b42b8e-9796-4297-88e7-de2ea3c296bc None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:19.018401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b42b8e-9796-4297-88e7-de2ea3c296bc None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:19.019108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b42b8e-9796-4297-88e7-de2ea3c296bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:19.020480 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b42b8e-9796-4297-88e7-de2ea3c296bc None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:19.105149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 452/881] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:10:18 2022] POST /identity/v3/services => generated 269 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:19.268387 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4a198-6111-44a4-9310-96f109b43e3f None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:19.268759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4a198-6111-44a4-9310-96f109b43e3f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:19.268977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4a198-6111-44a4-9310-96f109b43e3f None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:19.270707 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 430/882] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:19.276217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898898e-39b7-4c30-978e-a21d2f6e8ca6 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:19.276511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898898e-39b7-4c30-978e-a21d2f6e8ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:19.276794 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898898e-39b7-4c30-978e-a21d2f6e8ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:19.296050 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-a898898e-39b7-4c30-978e-a21d2f6e8ca6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:19.297275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a898898e-39b7-4c30-978e-a21d2f6e8ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:19.389437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 453/883] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:19.407605 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4a198-6111-44a4-9310-96f109b43e3f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:19.516474 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4a198-6111-44a4-9310-96f109b43e3f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:19.542447 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:19.546918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a95415-b44c-4a49-9b23-c36a22fa5884 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:19.547206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a95415-b44c-4a49-9b23-c36a22fa5884 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:19.547645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a95415-b44c-4a49-9b23-c36a22fa5884 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:19.549660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 454/884] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:10:19.553788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0847d9a-5af5-4be9-823b-09abf3066ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:19.554447 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0847d9a-5af5-4be9-823b-09abf3066ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:19.554645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8b48-4f09-4ec2-8770-4f93ee73b70c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:19.554766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0847d9a-5af5-4be9-823b-09abf3066ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:19.554937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8b48-4f09-4ec2-8770-4f93ee73b70c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:19.555043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0847d9a-5af5-4be9-823b-09abf3066ba9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:19.555222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8b48-4f09-4ec2-8770-4f93ee73b70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:19.555623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0847d9a-5af5-4be9-823b-09abf3066ba9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:19.557166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0847d9a-5af5-4be9-823b-09abf3066ba9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b0847d9a-5af5-4be9-823b-09abf3066ba9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:19.569559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:19.570896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:10:19.572014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:19.576093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 431/885] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:10:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:19.577282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0847d9a-5af5-4be9-823b-09abf3066ba9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:19.580740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-19cb8b48-4f09-4ec2-8770-4f93ee73b70c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:19.582345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-19cb8b48-4f09-4ec2-8770-4f93ee73b70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:19.669463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 455/886] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:19.689812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8b48-4f09-4ec2-8770-4f93ee73b70c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:19.706715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0847d9a-5af5-4be9-823b-09abf3066ba9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:19.723795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:19.751042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80aa6f6-0e76-4429-bafd-146952913ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:19.752098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80aa6f6-0e76-4429-bafd-146952913ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:19.752485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80aa6f6-0e76-4429-bafd-146952913ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:19.752894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80aa6f6-0e76-4429-bafd-146952913ae9 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:19.753708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80aa6f6-0e76-4429-bafd-146952913ae9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:19.754133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80aa6f6-0e76-4429-bafd-146952913ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:19.755612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80aa6f6-0e76-4429-bafd-146952913ae9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:19.761575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 432/887] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:10:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:19.768013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80aa6f6-0e76-4429-bafd-146952913ae9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:19.821573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8b48-4f09-4ec2-8770-4f93ee73b70c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:19.835114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:19.870335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298dbd-7fc8-491e-b68c-0b8e53347eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:19.871009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298dbd-7fc8-491e-b68c-0b8e53347eac None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:19.871299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298dbd-7fc8-491e-b68c-0b8e53347eac None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:19.871584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298dbd-7fc8-491e-b68c-0b8e53347eac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:19.887307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f298dbd-7fc8-491e-b68c-0b8e53347eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-3f298dbd-7fc8-491e-b68c-0b8e53347eac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:19.893917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:19.894962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:19.895796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:19.896433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80aa6f6-0e76-4429-bafd-146952913ae9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:19.903208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 456/888] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 28 06:10:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:19.904355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298dbd-7fc8-491e-b68c-0b8e53347eac None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:19.942896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97282d22-7802-4631-8187-44dd3db4d84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:19.943529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97282d22-7802-4631-8187-44dd3db4d84b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:19.944070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97282d22-7802-4631-8187-44dd3db4d84b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:19.944362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97282d22-7802-4631-8187-44dd3db4d84b None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:19.945067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97282d22-7802-4631-8187-44dd3db4d84b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:19.946314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97282d22-7802-4631-8187-44dd3db4d84b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:19.947806 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97282d22-7802-4631-8187-44dd3db4d84b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:19.957380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 433/889] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:10:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:19.961041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97282d22-7802-4631-8187-44dd3db4d84b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:20.014705 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:20.024455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298dbd-7fc8-491e-b68c-0b8e53347eac None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:20.061963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b9397-d1f4-4903-91bc-b88cc02df11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:20.063004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b9397-d1f4-4903-91bc-b88cc02df11b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:20.063790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b9397-d1f4-4903-91bc-b88cc02df11b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:20.064337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b9397-d1f4-4903-91bc-b88cc02df11b None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:20.065280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b9397-d1f4-4903-91bc-b88cc02df11b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:20.065825 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b9397-d1f4-4903-91bc-b88cc02df11b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:20.067399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b9397-d1f4-4903-91bc-b88cc02df11b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:20.081735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 457/890] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 28 06:10:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:20.084598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b9397-d1f4-4903-91bc-b88cc02df11b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:20.100434 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97282d22-7802-4631-8187-44dd3db4d84b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:20.124585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:20.154959 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723d876-4767-42bc-85a5-b0a7d99aeda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:20.155845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723d876-4767-42bc-85a5-b0a7d99aeda0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:20.156194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723d876-4767-42bc-85a5-b0a7d99aeda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:20.158294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723d876-4767-42bc-85a5-b0a7d99aeda0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:20.159498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723d876-4767-42bc-85a5-b0a7d99aeda0 None admin] RBAC: Adding query filter params (interface=public, service_id=fea9940629f345cfb6a17a4f3c36dbad, region_id=RegionOne) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:20.159839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723d876-4767-42bc-85a5-b0a7d99aeda0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:20.161549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723d876-4767-42bc-85a5-b0a7d99aeda0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:20.174962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 434/891] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 28 06:10:19 2022] GET /identity/v3/endpoints?service_id=fea9940629f345cfb6a17a4f3c36dbad&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:20.176847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723d876-4767-42bc-85a5-b0a7d99aeda0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:20.206787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b9397-d1f4-4903-91bc-b88cc02df11b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:20.246494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02a173-1b8d-440a-ba70-5ba47007072f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:20.246872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02a173-1b8d-440a-ba70-5ba47007072f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:20.247158 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02a173-1b8d-440a-ba70-5ba47007072f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:20.247431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02a173-1b8d-440a-ba70-5ba47007072f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:20.259511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02a173-1b8d-440a-ba70-5ba47007072f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:20.261373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02a173-1b8d-440a-ba70-5ba47007072f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-ec02a173-1b8d-440a-ba70-5ba47007072f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:20.277520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:20.278670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:20.279765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:20.279765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:20.279765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:20.279765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:20.280208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 458/892] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:20.281529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05abf93-4f1c-4bcd-8c4c-9e9dbb0873e2 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:20.281765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05abf93-4f1c-4bcd-8c4c-9e9dbb0873e2 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:20.282051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05abf93-4f1c-4bcd-8c4c-9e9dbb0873e2 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:20.283469 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 459/893] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:20.288280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05abf93-4f1c-4bcd-8c4c-9e9dbb0873e2 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:20.298580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:20.301513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723d876-4767-42bc-85a5-b0a7d99aeda0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:20.341726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8573b-143d-4827-adff-71c9a379addc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:20.342288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8573b-143d-4827-adff-71c9a379addc None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:20.342563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8573b-143d-4827-adff-71c9a379addc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:20.342925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8573b-143d-4827-adff-71c9a379addc None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:20.343491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8573b-143d-4827-adff-71c9a379addc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:20.344894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8573b-143d-4827-adff-71c9a379addc None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:20.356759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 435/894] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 28 06:10:20 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:10:20.358391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a489a-3ee7-4dd4-b042-e19acc5938a9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:20.359361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a489a-3ee7-4dd4-b042-e19acc5938a9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:20.359841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a489a-3ee7-4dd4-b042-e19acc5938a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:20.393371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d44a489a-3ee7-4dd4-b042-e19acc5938a9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:20.395024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d44a489a-3ee7-4dd4-b042-e19acc5938a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:20.428409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05abf93-4f1c-4bcd-8c4c-9e9dbb0873e2 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:20.449915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:20.480770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1badc-4382-4ab3-b220-00d3dd569ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:20.481795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1badc-4382-4ab3-b220-00d3dd569ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:20.482281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1badc-4382-4ab3-b220-00d3dd569ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:20.482759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1badc-4382-4ab3-b220-00d3dd569ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:20.483693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1badc-4382-4ab3-b220-00d3dd569ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:20.484197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1badc-4382-4ab3-b220-00d3dd569ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:20.485092 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 436/895] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:20.486119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1badc-4382-4ab3-b220-00d3dd569ea3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:20.501712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 460/896] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:20.504010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a489a-3ee7-4dd4-b042-e19acc5938a9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:20.509368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1badc-4382-4ab3-b220-00d3dd569ea3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:20.595612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:20.640843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a489a-3ee7-4dd4-b042-e19acc5938a9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:20.641901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1badc-4382-4ab3-b220-00d3dd569ea3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:20.687984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03619241-1d76-4932-9712-58cb4f9be55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:20.689633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03619241-1d76-4932-9712-58cb4f9be55c None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:20.689769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dcf83-b736-4871-9382-4bb861ea73d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:20.690263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03619241-1d76-4932-9712-58cb4f9be55c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:20.690733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03619241-1d76-4932-9712-58cb4f9be55c None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:20.691080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dcf83-b736-4871-9382-4bb861ea73d9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:20.691623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dcf83-b736-4871-9382-4bb861ea73d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:20.692030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03619241-1d76-4932-9712-58cb4f9be55c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:20.692182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dcf83-b736-4871-9382-4bb861ea73d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:20.693780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03619241-1d76-4932-9712-58cb4f9be55c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:20.700534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dcf83-b736-4871-9382-4bb861ea73d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:20.702094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dcf83-b736-4871-9382-4bb861ea73d9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-2b8dcf83-b736-4871-9382-4bb861ea73d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:20.714747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:20.715917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:20.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:20.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:20.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:20.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:20.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:20.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:20.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:20.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:20.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:20.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:20.718029 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 461/897] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 28 06:10:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:20.722237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dcf83-b736-4871-9382-4bb861ea73d9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:20.765025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 437/898] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:10:20 2022] POST /identity/v3/services => generated 241 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:20.846708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dcf83-b736-4871-9382-4bb861ea73d9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:20.877143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:20.894201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bde5e-bfef-4b6f-874e-c8661cf0eda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:20.895133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bde5e-bfef-4b6f-874e-c8661cf0eda7 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:20.895560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bde5e-bfef-4b6f-874e-c8661cf0eda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:20.895981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bde5e-bfef-4b6f-874e-c8661cf0eda7 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:20.896990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9bde5e-bfef-4b6f-874e-c8661cf0eda7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:20.897507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9bde5e-bfef-4b6f-874e-c8661cf0eda7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:20.899118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9bde5e-bfef-4b6f-874e-c8661cf0eda7 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:20.925444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 462/899] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 28 06:10:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:20.927311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03619241-1d76-4932-9712-58cb4f9be55c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:20.965132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:21.041177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03619241-1d76-4932-9712-58cb4f9be55c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:21.079098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471be01-5e6b-477f-98b7-a2ed39c44241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:21.080203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471be01-5e6b-477f-98b7-a2ed39c44241 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:21.080632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471be01-5e6b-477f-98b7-a2ed39c44241 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:21.081168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471be01-5e6b-477f-98b7-a2ed39c44241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:21.154897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471be01-5e6b-477f-98b7-a2ed39c44241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:21.157170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471be01-5e6b-477f-98b7-a2ed39c44241 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:21.193482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:21.237624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787eab55-5a64-418c-8c9a-559c6c7225aa None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:21.237910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787eab55-5a64-418c-8c9a-559c6c7225aa None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:21.238187 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787eab55-5a64-418c-8c9a-559c6c7225aa None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:21.239797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 463/900] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:21.244886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842890c-e45b-49eb-b6cf-cdd2333065c8 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:21.245174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842890c-e45b-49eb-b6cf-cdd2333065c8 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:21.245486 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842890c-e45b-49eb-b6cf-cdd2333065c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:21.264885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-c842890c-e45b-49eb-b6cf-cdd2333065c8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:21.266650 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c842890c-e45b-49eb-b6cf-cdd2333065c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:21.273413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 438/901] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:20 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:10:21.274929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471be01-5e6b-477f-98b7-a2ed39c44241 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:21.361766 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 464/902] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:21.388879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842890c-e45b-49eb-b6cf-cdd2333065c8 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:21.402114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471be01-5e6b-477f-98b7-a2ed39c44241 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:21.448195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd18ce-dc4d-46f1-ae24-dbcb24df1825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:21.449189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd18ce-dc4d-46f1-ae24-dbcb24df1825 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:21.449747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd18ce-dc4d-46f1-ae24-dbcb24df1825 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:21.450229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd18ce-dc4d-46f1-ae24-dbcb24df1825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:21.453048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:21.463226 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd18ce-dc4d-46f1-ae24-dbcb24df1825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:21.471061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd18ce-dc4d-46f1-ae24-dbcb24df1825 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-cbdd18ce-dc4d-46f1-ae24-dbcb24df1825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:21.483375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:21.484292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:21.485868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:21.485868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:21.485868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:21.485868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:21.495079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 439/903] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:21.499451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd18ce-dc4d-46f1-ae24-dbcb24df1825 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:21.522001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842890c-e45b-49eb-b6cf-cdd2333065c8 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:21.537492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:21.571955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b7237-0b2a-4530-b53e-23623a5918ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:21.572627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b7237-0b2a-4530-b53e-23623a5918ff None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:21.573281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b7237-0b2a-4530-b53e-23623a5918ff None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:21.573611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b7237-0b2a-4530-b53e-23623a5918ff None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:21.574323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b7237-0b2a-4530-b53e-23623a5918ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:21.576378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b7237-0b2a-4530-b53e-23623a5918ff None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-cc0b7237-0b2a-4530-b53e-23623a5918ff None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:21.590581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:21.591605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:21.598231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 465/904] 10.222.0.69 () {56 vars in 1132 bytes} [Thu Jul 28 06:10:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:21.601077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b7237-0b2a-4530-b53e-23623a5918ff None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:21.630096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd18ce-dc4d-46f1-ae24-dbcb24df1825 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:21.673411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4684c-27bc-4dc3-bad6-fddc5b3323c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:21.674085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4684c-27bc-4dc3-bad6-fddc5b3323c9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:21.674378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4684c-27bc-4dc3-bad6-fddc5b3323c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:21.674660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4684c-27bc-4dc3-bad6-fddc5b3323c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:21.675381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4684c-27bc-4dc3-bad6-fddc5b3323c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:21.675686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4684c-27bc-4dc3-bad6-fddc5b3323c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:21.677974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4684c-27bc-4dc3-bad6-fddc5b3323c9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:21.699456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 440/905] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:21.702280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4684c-27bc-4dc3-bad6-fddc5b3323c9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:21.731936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b7237-0b2a-4530-b53e-23623a5918ff None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:21.769997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:21.772634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a407cd3-4413-48fe-aac1-24093ef440af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:21.773614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a407cd3-4413-48fe-aac1-24093ef440af None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:21.774052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a407cd3-4413-48fe-aac1-24093ef440af None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:21.774498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a407cd3-4413-48fe-aac1-24093ef440af None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:21.775454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a407cd3-4413-48fe-aac1-24093ef440af None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:21.775915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a407cd3-4413-48fe-aac1-24093ef440af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:21.777698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a407cd3-4413-48fe-aac1-24093ef440af None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:21.783434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 466/906] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:21.789741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a407cd3-4413-48fe-aac1-24093ef440af None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:21.816363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4684c-27bc-4dc3-bad6-fddc5b3323c9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:21.833953 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:21.852197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715ec31-af5a-4fca-8bd5-1ea85131777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:21.855016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715ec31-af5a-4fca-8bd5-1ea85131777b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:21.855016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715ec31-af5a-4fca-8bd5-1ea85131777b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:21.855016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715ec31-af5a-4fca-8bd5-1ea85131777b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:21.862717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715ec31-af5a-4fca-8bd5-1ea85131777b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:21.864827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715ec31-af5a-4fca-8bd5-1ea85131777b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8715ec31-af5a-4fca-8bd5-1ea85131777b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:21.883250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:21.884221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:21.885116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:21.885116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:21.885116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:21.885116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:21.885116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:21.885116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:21.885116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:21.885116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:21.887454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 441/907] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:21.891554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715ec31-af5a-4fca-8bd5-1ea85131777b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:21.907986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a407cd3-4413-48fe-aac1-24093ef440af None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:21.946494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d294d-2fee-429a-b145-4a24fafe46a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:21.947131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d294d-2fee-429a-b145-4a24fafe46a7 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:21.947523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d294d-2fee-429a-b145-4a24fafe46a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:21.947801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d294d-2fee-429a-b145-4a24fafe46a7 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:21.948497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d294d-2fee-429a-b145-4a24fafe46a7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:21.948809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d294d-2fee-429a-b145-4a24fafe46a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:21.950398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d294d-2fee-429a-b145-4a24fafe46a7 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:21.960589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 467/908] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:21.963215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d294d-2fee-429a-b145-4a24fafe46a7 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:22.022660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715ec31-af5a-4fca-8bd5-1ea85131777b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:22.032718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:22.073239 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdc022-555c-4b1e-8add-b1af741a776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:22.073899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdc022-555c-4b1e-8add-b1af741a776b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:22.074195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdc022-555c-4b1e-8add-b1af741a776b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:22.074556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdc022-555c-4b1e-8add-b1af741a776b None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:22.075331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdc022-555c-4b1e-8add-b1af741a776b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:22.075640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdc022-555c-4b1e-8add-b1af741a776b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:22.077098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdc022-555c-4b1e-8add-b1af741a776b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:22.091790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d294d-2fee-429a-b145-4a24fafe46a7 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:22.093350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 442/909] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:22.098372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:22.099205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdc022-555c-4b1e-8add-b1af741a776b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:22.136801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d221456-6e5e-4697-990a-077f899749a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:22.137466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d221456-6e5e-4697-990a-077f899749a9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:22.137798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d221456-6e5e-4697-990a-077f899749a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:22.138081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d221456-6e5e-4697-990a-077f899749a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:22.138955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d221456-6e5e-4697-990a-077f899749a9 None admin] RBAC: Adding query filter params (interface=public, service_id=842df33d95e94c7ba0a538e94e9107f6, region_id=RegionOne) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:22.139273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d221456-6e5e-4697-990a-077f899749a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:22.140657 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d221456-6e5e-4697-990a-077f899749a9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:22.151081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 468/910] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 28 06:10:21 2022] GET /identity/v3/endpoints?service_id=842df33d95e94c7ba0a538e94e9107f6&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:22.154260 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d221456-6e5e-4697-990a-077f899749a9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:22.221875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdc022-555c-4b1e-8add-b1af741a776b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:22.258798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4b096-22f3-439d-8210-7b801a6425e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:22.259427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4b096-22f3-439d-8210-7b801a6425e9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:22.259766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4b096-22f3-439d-8210-7b801a6425e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:22.259979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4b096-22f3-439d-8210-7b801a6425e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:22.261043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4b096-22f3-439d-8210-7b801a6425e9 None admin] RBAC: Adding query filter params (role.id=7945ffd9805a4831932cab7339f7993c, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=39c6f0ee9b78413eb8c5efc7adf145b6) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:22.261389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4b096-22f3-439d-8210-7b801a6425e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:22.262844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4b096-22f3-439d-8210-7b801a6425e9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:22.269352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 443/911] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:22 2022] GET /identity/v3/role_assignments?user.id=39c6f0ee9b78413eb8c5efc7adf145b6&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=7945ffd9805a4831932cab7339f7993c => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:22.290196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d221456-6e5e-4697-990a-077f899749a9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:22.329512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1eed61-ba7d-48c6-8df8-542f25dcbc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:22.330509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1eed61-ba7d-48c6-8df8-542f25dcbc30 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:22.330987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1eed61-ba7d-48c6-8df8-542f25dcbc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:22.331500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1eed61-ba7d-48c6-8df8-542f25dcbc30 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:22.332433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1eed61-ba7d-48c6-8df8-542f25dcbc30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:22.334104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1eed61-ba7d-48c6-8df8-542f25dcbc30 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:22.341656 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 469/912] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 28 06:10:22 2022] GET /identity/v3/services => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:10:22.532687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eda441-774e-4247-aeb0-8f0a62498eb1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:22.533147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eda441-774e-4247-aeb0-8f0a62498eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:22.533465 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eda441-774e-4247-aeb0-8f0a62498eb1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:22.535050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 444/913] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:22.540673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697438f8-1d0f-41ef-9f99-40991de56ec7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:22.540962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697438f8-1d0f-41ef-9f99-40991de56ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:22.541185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697438f8-1d0f-41ef-9f99-40991de56ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:22.561545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-697438f8-1d0f-41ef-9f99-40991de56ec7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:22.563320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-697438f8-1d0f-41ef-9f99-40991de56ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:22.589345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:22.637683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 470/914] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:22.661964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eda441-774e-4247-aeb0-8f0a62498eb1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:22.696112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:22.773612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eda441-774e-4247-aeb0-8f0a62498eb1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:22.815932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9cdc1-2e1a-4bf0-b2b7-4158216c8ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:22.816500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9cdc1-2e1a-4bf0-b2b7-4158216c8ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:22.816783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9cdc1-2e1a-4bf0-b2b7-4158216c8ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:22.817059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9cdc1-2e1a-4bf0-b2b7-4158216c8ba2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:22.817832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9cdc1-2e1a-4bf0-b2b7-4158216c8ba2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:22.819270 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9cdc1-2e1a-4bf0-b2b7-4158216c8ba2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-bbe9cdc1-2e1a-4bf0-b2b7-4158216c8ba2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:22.829397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:22.830413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:22.831237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 445/915] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:10:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:22.837828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697438f8-1d0f-41ef-9f99-40991de56ec7 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:22.912290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152c945-6abf-4b3f-9e45-b230c36238b6 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:22.912581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152c945-6abf-4b3f-9e45-b230c36238b6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:22.912859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152c945-6abf-4b3f-9e45-b230c36238b6 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:22.914510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 446/916] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:10:22.920483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37a73e-920d-4774-9efd-74dd368f321a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:22.920896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37a73e-920d-4774-9efd-74dd368f321a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:22.921451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37a73e-920d-4774-9efd-74dd368f321a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:22.942132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4c37a73e-920d-4774-9efd-74dd368f321a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:22.943690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4c37a73e-920d-4774-9efd-74dd368f321a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:22.953639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697438f8-1d0f-41ef-9f99-40991de56ec7 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:22.999557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffcfa0-b4e9-4d8b-baaa-a813bd54744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:23.000412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffcfa0-b4e9-4d8b-baaa-a813bd54744a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:23.000707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffcfa0-b4e9-4d8b-baaa-a813bd54744a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:23.000995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffcfa0-b4e9-4d8b-baaa-a813bd54744a None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:23.002115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffcfa0-b4e9-4d8b-baaa-a813bd54744a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:23.002736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffcfa0-b4e9-4d8b-baaa-a813bd54744a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:23.004356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffcfa0-b4e9-4d8b-baaa-a813bd54744a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:23.015518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 471/917] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:10:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:23.017004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffcfa0-b4e9-4d8b-baaa-a813bd54744a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:23.028492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 447/918] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:23.049715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:23.051923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37a73e-920d-4774-9efd-74dd368f321a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:23.084312 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:23.144861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffcfa0-b4e9-4d8b-baaa-a813bd54744a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:23.188884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37a73e-920d-4774-9efd-74dd368f321a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:23.193494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd0632-c085-47a4-871c-eb5649e45cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:23.194287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd0632-c085-47a4-871c-eb5649e45cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:23.194725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd0632-c085-47a4-871c-eb5649e45cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:23.195145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd0632-c085-47a4-871c-eb5649e45cc8 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:23.195966 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd0632-c085-47a4-871c-eb5649e45cc8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:23.196438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd0632-c085-47a4-871c-eb5649e45cc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:23.198488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd0632-c085-47a4-871c-eb5649e45cc8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:23.205362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 472/919] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 28 06:10:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:23.210796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd0632-c085-47a4-871c-eb5649e45cc8 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:23.233150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933961f3-40ee-4762-80bf-694bc6974e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:23.234158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933961f3-40ee-4762-80bf-694bc6974e93 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:23.234449 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933961f3-40ee-4762-80bf-694bc6974e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:23.235346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933961f3-40ee-4762-80bf-694bc6974e93 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:23.236009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933961f3-40ee-4762-80bf-694bc6974e93 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:23.237422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933961f3-40ee-4762-80bf-694bc6974e93 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-933961f3-40ee-4762-80bf-694bc6974e93 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:23.249245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:23.250938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:23.257161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 448/920] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:23.258247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933961f3-40ee-4762-80bf-694bc6974e93 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:23.327333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:23.344251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd0632-c085-47a4-871c-eb5649e45cc8 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:23.364237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:23.378417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933961f3-40ee-4762-80bf-694bc6974e93 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:23.392967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d4a1f-0c17-40eb-aa47-80262bf4bc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:23.394351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d4a1f-0c17-40eb-aa47-80262bf4bc6a None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:23.394841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d4a1f-0c17-40eb-aa47-80262bf4bc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:23.395695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d4a1f-0c17-40eb-aa47-80262bf4bc6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:23.396901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d4a1f-0c17-40eb-aa47-80262bf4bc6a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:23.398996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d4a1f-0c17-40eb-aa47-80262bf4bc6a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:23.423501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b8456-182a-41bb-9fb6-b356e01c99bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:23.424182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b8456-182a-41bb-9fb6-b356e01c99bc None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:23.425673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b8456-182a-41bb-9fb6-b356e01c99bc None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:23.425673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b8456-182a-41bb-9fb6-b356e01c99bc None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:23.425673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b8456-182a-41bb-9fb6-b356e01c99bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:23.425930 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b8456-182a-41bb-9fb6-b356e01c99bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:23.427121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b8456-182a-41bb-9fb6-b356e01c99bc None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:23.436993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 449/921] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 28 06:10:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:23.437900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b8456-182a-41bb-9fb6-b356e01c99bc None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:23.491370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 473/922] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:10:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:10:23.574012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b8456-182a-41bb-9fb6-b356e01c99bc None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:23.623040 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3a99d-2923-4507-85df-0a9e4f8ceaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:23.623602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3a99d-2923-4507-85df-0a9e4f8ceaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:23.623892 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3a99d-2923-4507-85df-0a9e4f8ceaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:23.624176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3a99d-2923-4507-85df-0a9e4f8ceaf8 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:23.624886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3a99d-2923-4507-85df-0a9e4f8ceaf8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:23.625228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3a99d-2923-4507-85df-0a9e4f8ceaf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:23.626909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3a99d-2923-4507-85df-0a9e4f8ceaf8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:23.633443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 450/923] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 28 06:10:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:23.639320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d4a1f-0c17-40eb-aa47-80262bf4bc6a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:23.755668 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d4a1f-0c17-40eb-aa47-80262bf4bc6a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:23.794430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e630f-5a77-4fba-9d90-86b1ec0996bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:23.795106 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e630f-5a77-4fba-9d90-86b1ec0996bf None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:23.795395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e630f-5a77-4fba-9d90-86b1ec0996bf None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:23.795672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e630f-5a77-4fba-9d90-86b1ec0996bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:23.796461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e630f-5a77-4fba-9d90-86b1ec0996bf None admin] RBAC: Adding query filter params (interface=public, service_id=cdc63d9cb9794068a2778b41f55ca6db, region_id=RegionOne) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:23.796777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e630f-5a77-4fba-9d90-86b1ec0996bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:23.798920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e630f-5a77-4fba-9d90-86b1ec0996bf None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:23.805440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 474/924] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 28 06:10:23 2022] GET /identity/v3/endpoints?service_id=cdc63d9cb9794068a2778b41f55ca6db&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:23.811935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3a99d-2923-4507-85df-0a9e4f8ceaf8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:23.832422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:23.931099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3a99d-2923-4507-85df-0a9e4f8ceaf8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:23.967776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f3850-2eba-4a84-8c96-a519222d114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:23.968423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f3850-2eba-4a84-8c96-a519222d114a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:23.968711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f3850-2eba-4a84-8c96-a519222d114a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:23.969565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f3850-2eba-4a84-8c96-a519222d114a None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:23.970250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f3850-2eba-4a84-8c96-a519222d114a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:23.971681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f3850-2eba-4a84-8c96-a519222d114a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:23.978849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 451/925] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 28 06:10:23 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:10:24.450528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa41e85-7d62-44a6-90ac-85fe7c21e3f9 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:24.451075 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa41e85-7d62-44a6-90ac-85fe7c21e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:24.451075 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa41e85-7d62-44a6-90ac-85fe7c21e3f9 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:24.452677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 475/926] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:24.459158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd454de-adec-497c-84f3-858989fae6ac None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:24.459452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd454de-adec-497c-84f3-858989fae6ac None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:24.459740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd454de-adec-497c-84f3-858989fae6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:24.479706 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4fd454de-adec-497c-84f3-858989fae6ac None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:24.483580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4fd454de-adec-497c-84f3-858989fae6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:24.514953 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547401e0-2978-40af-9241-f3eb7479e151 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:24.515289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547401e0-2978-40af-9241-f3eb7479e151 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:24.515518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547401e0-2978-40af-9241-f3eb7479e151 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:24.517762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 476/927] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:24.523746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa921-611b-42c1-9ff7-8f5a616636e9 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:24.523878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa921-611b-42c1-9ff7-8f5a616636e9 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:24.524070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa921-611b-42c1-9ff7-8f5a616636e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:24.544256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-320aa921-611b-42c1-9ff7-8f5a616636e9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:24.545662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-320aa921-611b-42c1-9ff7-8f5a616636e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:24.549564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:24.559589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 452/928] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:24.582991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd454de-adec-497c-84f3-858989fae6ac None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:24.591508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:24.633513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 477/929] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:24.661580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aa921-611b-42c1-9ff7-8f5a616636e9 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:24.708764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd454de-adec-497c-84f3-858989fae6ac None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:24.745369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bb1e1-4b73-4f59-a7f0-c0a46694595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:24.746102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bb1e1-4b73-4f59-a7f0-c0a46694595a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:24.746396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bb1e1-4b73-4f59-a7f0-c0a46694595a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:24.746678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bb1e1-4b73-4f59-a7f0-c0a46694595a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:24.764166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bb1e1-4b73-4f59-a7f0-c0a46694595a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-053bb1e1-4b73-4f59-a7f0-c0a46694595a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:24.770627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:24.771666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:24.772511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:24.779976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aa921-611b-42c1-9ff7-8f5a616636e9 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:24.780317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 453/930] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 28 06:10:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:24.785346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bb1e1-4b73-4f59-a7f0-c0a46694595a None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:24.818603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f7768-364a-4c0a-83be-af4af0bd84b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:24.819284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f7768-364a-4c0a-83be-af4af0bd84b8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:24.819575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f7768-364a-4c0a-83be-af4af0bd84b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:24.819861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f7768-364a-4c0a-83be-af4af0bd84b8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:24.820526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f7768-364a-4c0a-83be-af4af0bd84b8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:24.822020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f7768-364a-4c0a-83be-af4af0bd84b8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-1a7f7768-364a-4c0a-83be-af4af0bd84b8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:24.834093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:24.835270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:24.836112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 478/931] 10.222.0.69 () {56 vars in 1132 bytes} [Thu Jul 28 06:10:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:24.842218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f7768-364a-4c0a-83be-af4af0bd84b8 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:24.846437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:24.866599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:24.904870 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bb1e1-4b73-4f59-a7f0-c0a46694595a None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:24.942992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87767a4-03d1-4602-80ff-431c1953c034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:24.943965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87767a4-03d1-4602-80ff-431c1953c034 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:24.944460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87767a4-03d1-4602-80ff-431c1953c034 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:24.945029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87767a4-03d1-4602-80ff-431c1953c034 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:24.946154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87767a4-03d1-4602-80ff-431c1953c034 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:24.946676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87767a4-03d1-4602-80ff-431c1953c034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:24.948367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87767a4-03d1-4602-80ff-431c1953c034 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:24.951962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f7768-364a-4c0a-83be-af4af0bd84b8 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:24.962370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 454/932] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 28 06:10:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:24.963700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87767a4-03d1-4602-80ff-431c1953c034 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:24.989981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be6632-11a6-4623-8229-9fdb25af21ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:24.990546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be6632-11a6-4623-8229-9fdb25af21ba None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:24.990814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be6632-11a6-4623-8229-9fdb25af21ba None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:24.991100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be6632-11a6-4623-8229-9fdb25af21ba None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:24.991797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be6632-11a6-4623-8229-9fdb25af21ba None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:24.992110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be6632-11a6-4623-8229-9fdb25af21ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:24.993766 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be6632-11a6-4623-8229-9fdb25af21ba None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:25.003628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 479/933] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:25.004876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be6632-11a6-4623-8229-9fdb25af21ba None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:25.075834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:25.076303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87767a4-03d1-4602-80ff-431c1953c034 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:25.106055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:25.116096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dca244-0435-4a83-a300-c3f42d2ebcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:25.116751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dca244-0435-4a83-a300-c3f42d2ebcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:25.117037 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dca244-0435-4a83-a300-c3f42d2ebcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:25.117351 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dca244-0435-4a83-a300-c3f42d2ebcd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:25.120431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be6632-11a6-4623-8229-9fdb25af21ba None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:25.136466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dca244-0435-4a83-a300-c3f42d2ebcd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:25.138609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dca244-0435-4a83-a300-c3f42d2ebcd5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:25.160998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2e790-5c97-4751-abe2-df1b4537c96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:25.161681 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2e790-5c97-4751-abe2-df1b4537c96d None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:25.162043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2e790-5c97-4751-abe2-df1b4537c96d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:25.162401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2e790-5c97-4751-abe2-df1b4537c96d None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:25.163217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2e790-5c97-4751-abe2-df1b4537c96d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:25.163603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2e790-5c97-4751-abe2-df1b4537c96d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c1dca244-0435-4a83-a300-c3f42d2ebcd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:25.164115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:25.164962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:25.166054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:25.166054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:25.166054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:25.166054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:25.166273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2e790-5c97-4751-abe2-df1b4537c96d None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:25.168565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 455/934] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:25.174505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dca244-0435-4a83-a300-c3f42d2ebcd5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:25.178498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 480/935] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:25.180284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2e790-5c97-4751-abe2-df1b4537c96d None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:25.306340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dca244-0435-4a83-a300-c3f42d2ebcd5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:25.307375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2e790-5c97-4751-abe2-df1b4537c96d None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:25.342263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:25.348634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030b378-0e05-4cfb-b12c-a3336abdd17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:25.348634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030b378-0e05-4cfb-b12c-a3336abdd17c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:25.348634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030b378-0e05-4cfb-b12c-a3336abdd17c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:25.348634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030b378-0e05-4cfb-b12c-a3336abdd17c None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:25.348993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030b378-0e05-4cfb-b12c-a3336abdd17c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:25.348993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030b378-0e05-4cfb-b12c-a3336abdd17c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:25.349875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030b378-0e05-4cfb-b12c-a3336abdd17c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:25.356345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3089edd-6890-4715-bd44-7a1bf4f5d3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:25.357223 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3089edd-6890-4715-bd44-7a1bf4f5d3f7 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:25.357734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3089edd-6890-4715-bd44-7a1bf4f5d3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:25.358192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3089edd-6890-4715-bd44-7a1bf4f5d3f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:25.359116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3089edd-6890-4715-bd44-7a1bf4f5d3f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:25.360748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3089edd-6890-4715-bd44-7a1bf4f5d3f7 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:25.364747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 456/936] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:25.373334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030b378-0e05-4cfb-b12c-a3336abdd17c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:25.418131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:25.469911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 481/937] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:10:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:25.489240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030b378-0e05-4cfb-b12c-a3336abdd17c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:25.540019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd7c5e-4904-4cc8-ba19-00a247637289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:25.540903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd7c5e-4904-4cc8-ba19-00a247637289 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:25.541255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd7c5e-4904-4cc8-ba19-00a247637289 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:25.541579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd7c5e-4904-4cc8-ba19-00a247637289 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:25.556799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd7c5e-4904-4cc8-ba19-00a247637289 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:25.558699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd7c5e-4904-4cc8-ba19-00a247637289 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-dcbd7c5e-4904-4cc8-ba19-00a247637289 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:25.573232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:25.574284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:25.575236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 457/938] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 28 06:10:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:25.581168 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3089edd-6890-4715-bd44-7a1bf4f5d3f7 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:25.701144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3089edd-6890-4715-bd44-7a1bf4f5d3f7 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:25.741454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f2354-35eb-46d4-a16a-e6f8074e3b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:25.742149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f2354-35eb-46d4-a16a-e6f8074e3b14 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:25.742439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f2354-35eb-46d4-a16a-e6f8074e3b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:25.742723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f2354-35eb-46d4-a16a-e6f8074e3b14 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:25.743542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f2354-35eb-46d4-a16a-e6f8074e3b14 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:25.743858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f2354-35eb-46d4-a16a-e6f8074e3b14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:25.745330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f2354-35eb-46d4-a16a-e6f8074e3b14 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:25.760953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc9935-bc2a-492b-b191-d5489a9a2d76 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:25.761259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc9935-bc2a-492b-b191-d5489a9a2d76 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:25.761547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc9935-bc2a-492b-b191-d5489a9a2d76 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:25.763059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 458/939] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:10:25.768998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 482/940] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 28 06:10:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:25.769904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e5a18-e07b-4e10-b8d5-241962f1fa82 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:25.770265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e5a18-e07b-4e10-b8d5-241962f1fa82 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:25.770637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e5a18-e07b-4e10-b8d5-241962f1fa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:25.774208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f2354-35eb-46d4-a16a-e6f8074e3b14 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:25.791366 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-265e5a18-e07b-4e10-b8d5-241962f1fa82 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:25.792951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-265e5a18-e07b-4e10-b8d5-241962f1fa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:25.834861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:25.878207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 459/941] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:25.901627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e5a18-e07b-4e10-b8d5-241962f1fa82 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:25.907210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f2354-35eb-46d4-a16a-e6f8074e3b14 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:25.936104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:25.945920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221cf6e-cec9-4209-b7b6-72249f129c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:25.946584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221cf6e-cec9-4209-b7b6-72249f129c41 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:25.946958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221cf6e-cec9-4209-b7b6-72249f129c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:25.947284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221cf6e-cec9-4209-b7b6-72249f129c41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:25.961773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221cf6e-cec9-4209-b7b6-72249f129c41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:25.963165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221cf6e-cec9-4209-b7b6-72249f129c41 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-d221cf6e-cec9-4209-b7b6-72249f129c41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:25.983348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:25.984336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:25.986238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:25.986238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:25.986238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:25.986238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:25.986238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 483/942] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:25.990553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221cf6e-cec9-4209-b7b6-72249f129c41 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:26.034700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e5a18-e07b-4e10-b8d5-241962f1fa82 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:26.081982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17d609-09ae-4ae4-909d-efa17e2df31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:26.082895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17d609-09ae-4ae4-909d-efa17e2df31e None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.083902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17d609-09ae-4ae4-909d-efa17e2df31e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.084333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17d609-09ae-4ae4-909d-efa17e2df31e None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.085164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17d609-09ae-4ae4-909d-efa17e2df31e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:26.087258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17d609-09ae-4ae4-909d-efa17e2df31e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:26.091550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: WARNING keystone.server.flask.common [None req-ec17d609-09ae-4ae4-909d-efa17e2df31e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 06:10:26.106471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:26.113906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221cf6e-cec9-4209-b7b6-72249f129c41 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:26.136989 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.sql.core [None req-ec17d609-09ae-4ae4-909d-efa17e2df31e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 06:10:26.136989 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 06:10:26.136989 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [parameters: {'id': '9128647782e945f1a0377d1d76b52281', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 06:10:26.136989 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80318) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-ec17d609-09ae-4ae4-909d-efa17e2df31e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9128647782e945f1a0377d1d76b52281.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9128647782e945f1a0377d1d76b52281. Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:10:26.145549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result.read() Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 06:10:26.147679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application yield resource Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise self.value Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application session.commit() Jul 28 06:10:26.150161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.session.flush() Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application compat.raise_( Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 06:10:26.152232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 06:10:26.158187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application result.read() Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:10:26.159855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [parameters: {'id': '9128647782e945f1a0377d1d76b52281', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:26.160856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:26.162859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9128647782e945f1a0377d1d76b52281. Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 460/943] 10.222.0.69 () {60 vars in 1140 bytes} [Thu Jul 28 06:10:25 2022] POST /identity/v3/projects => generated 262 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3719c-8887-4f42-8f92-4c4b53efc890 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.164868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3719c-8887-4f42-8f92-4c4b53efc890 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.166479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3719c-8887-4f42-8f92-4c4b53efc890 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.166479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 461/944] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:26.166479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3719c-8887-4f42-8f92-4c4b53efc890 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:26.166892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753c358-1b02-4cd4-b898-7e9fb3ec874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:26.166892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753c358-1b02-4cd4-b898-7e9fb3ec874f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.166892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753c358-1b02-4cd4-b898-7e9fb3ec874f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.166892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753c358-1b02-4cd4-b898-7e9fb3ec874f None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.167479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753c358-1b02-4cd4-b898-7e9fb3ec874f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:26.167479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753c358-1b02-4cd4-b898-7e9fb3ec874f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:26.167479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753c358-1b02-4cd4-b898-7e9fb3ec874f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:26.191318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 484/945] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:26.191318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea405c3e-fde7-4451-9e59-345647457edf None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.191318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea405c3e-fde7-4451-9e59-345647457edf None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.191318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea405c3e-fde7-4451-9e59-345647457edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.213512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-ea405c3e-fde7-4451-9e59-345647457edf None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:26.215156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ea405c3e-fde7-4451-9e59-345647457edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:26.257378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:26.279714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3719c-8887-4f42-8f92-4c4b53efc890 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:26.300557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 485/946] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:26.301446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea405c3e-fde7-4451-9e59-345647457edf None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:26.319179 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4307c5-11e5-420e-a207-1eef4a25e730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:26.320286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4307c5-11e5-420e-a207-1eef4a25e730 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.320879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4307c5-11e5-420e-a207-1eef4a25e730 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.321441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4307c5-11e5-420e-a207-1eef4a25e730 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.331242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4307c5-11e5-420e-a207-1eef4a25e730 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:26.333168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4307c5-11e5-420e-a207-1eef4a25e730 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-cc4307c5-11e5-420e-a207-1eef4a25e730 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:26.348812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:26.350004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:26.350900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:26.350900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:26.350900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:26.350900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:26.350900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:26.350900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:26.350900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:26.350900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.353041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 462/947] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:26.353975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4307c5-11e5-420e-a207-1eef4a25e730 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:26.402406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:26.420017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea405c3e-fde7-4451-9e59-345647457edf None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:26.456413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e354b-28f5-4fea-be8c-17689b87d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:26.457072 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e354b-28f5-4fea-be8c-17689b87d638 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.457389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e354b-28f5-4fea-be8c-17689b87d638 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.457676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e354b-28f5-4fea-be8c-17689b87d638 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.464577 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4307c5-11e5-420e-a207-1eef4a25e730 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:26.464702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e354b-28f5-4fea-be8c-17689b87d638 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:26.466068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e354b-28f5-4fea-be8c-17689b87d638 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-d02e354b-28f5-4fea-be8c-17689b87d638 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:26.479520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:26.486571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:26.487792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:26.487792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:26.487792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:26.487792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:26.487792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:26.487792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:26.487792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:26.487792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:26.487792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 486/948] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:26.487792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e354b-28f5-4fea-be8c-17689b87d638 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:26.512822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a706bf7-17c0-496d-8470-a6124e7612a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:26.513522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a706bf7-17c0-496d-8470-a6124e7612a8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.514160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a706bf7-17c0-496d-8470-a6124e7612a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.514452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a706bf7-17c0-496d-8470-a6124e7612a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.515379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a706bf7-17c0-496d-8470-a6124e7612a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:26.518001 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a706bf7-17c0-496d-8470-a6124e7612a8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8a706bf7-17c0-496d-8470-a6124e7612a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:26.529887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:26.530891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:26.531760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 463/949] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:26.532542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a706bf7-17c0-496d-8470-a6124e7612a8 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:26.547812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:26.605245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e354b-28f5-4fea-be8c-17689b87d638 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:26.647977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46008a6-be7a-48bf-8cce-60b4d90e081c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:26.648618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46008a6-be7a-48bf-8cce-60b4d90e081c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.649119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46008a6-be7a-48bf-8cce-60b4d90e081c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.649607 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46008a6-be7a-48bf-8cce-60b4d90e081c None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.650630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46008a6-be7a-48bf-8cce-60b4d90e081c None admin] RBAC: Adding query filter params (name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:26.651098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46008a6-be7a-48bf-8cce-60b4d90e081c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:26.652398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46008a6-be7a-48bf-8cce-60b4d90e081c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:26.653855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a706bf7-17c0-496d-8470-a6124e7612a8 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:26.671089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 487/950] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:26.672957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46008a6-be7a-48bf-8cce-60b4d90e081c None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:26.682582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:26.709519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51861c3b-3499-40f9-a8c2-a0e59144f8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:26.710166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51861c3b-3499-40f9-a8c2-a0e59144f8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.710451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51861c3b-3499-40f9-a8c2-a0e59144f8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.710728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51861c3b-3499-40f9-a8c2-a0e59144f8f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.711520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51861c3b-3499-40f9-a8c2-a0e59144f8f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:26.711831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51861c3b-3499-40f9-a8c2-a0e59144f8f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:26.713378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51861c3b-3499-40f9-a8c2-a0e59144f8f1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:26.728208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 464/951] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:26.733769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51861c3b-3499-40f9-a8c2-a0e59144f8f1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:26.807216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46008a6-be7a-48bf-8cce-60b4d90e081c None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:26.814598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:26.851335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56719225-d39f-4e66-9302-9e95f4ef9219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:26.851976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56719225-d39f-4e66-9302-9e95f4ef9219 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.852249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56719225-d39f-4e66-9302-9e95f4ef9219 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.852697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56719225-d39f-4e66-9302-9e95f4ef9219 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.853272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56719225-d39f-4e66-9302-9e95f4ef9219 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:26.853561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56719225-d39f-4e66-9302-9e95f4ef9219 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:26.855798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56719225-d39f-4e66-9302-9e95f4ef9219 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:26.858633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51861c3b-3499-40f9-a8c2-a0e59144f8f1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:26.862230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 488/952] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 28 06:10:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:26.866854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56719225-d39f-4e66-9302-9e95f4ef9219 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:26.901128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42e38f-d6ff-4904-b723-bb9d5e54c0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:26.901856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42e38f-d6ff-4904-b723-bb9d5e54c0ec None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:26.902151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42e38f-d6ff-4904-b723-bb9d5e54c0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:26.902436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42e38f-d6ff-4904-b723-bb9d5e54c0ec None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/39c6f0ee9b78413eb8c5efc7adf145b6/roles/7945ffd9805a4831932cab7339f7993c` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:26.904803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42e38f-d6ff-4904-b723-bb9d5e54c0ec None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=39c6f0ee9b78413eb8c5efc7adf145b6, role_id=7945ffd9805a4831932cab7339f7993c)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:26.906972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42e38f-d6ff-4904-b723-bb9d5e54c0ec None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:26.948233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:26.983010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 465/953] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:10:26 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/39c6f0ee9b78413eb8c5efc7adf145b6/roles/7945ffd9805a4831932cab7339f7993c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:10:26.992322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56719225-d39f-4e66-9302-9e95f4ef9219 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:27.036135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770a896-f8da-42dd-8d9b-de28d6e56911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:27.036787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770a896-f8da-42dd-8d9b-de28d6e56911 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:27.037131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770a896-f8da-42dd-8d9b-de28d6e56911 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:27.037440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770a896-f8da-42dd-8d9b-de28d6e56911 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:27.038194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770a896-f8da-42dd-8d9b-de28d6e56911 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:27.038506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770a896-f8da-42dd-8d9b-de28d6e56911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:27.039929 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770a896-f8da-42dd-8d9b-de28d6e56911 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:27.046276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 489/954] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 28 06:10:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:27.052407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42e38f-d6ff-4904-b723-bb9d5e54c0ec None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:27.174529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42e38f-d6ff-4904-b723-bb9d5e54c0ec None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:27.202783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:27.212143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547d7c2-789a-4bb2-9693-bbf914e39e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:27.212817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547d7c2-789a-4bb2-9693-bbf914e39e4d None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:27.213104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547d7c2-789a-4bb2-9693-bbf914e39e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:27.213424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547d7c2-789a-4bb2-9693-bbf914e39e4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:27.214154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547d7c2-789a-4bb2-9693-bbf914e39e4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:27.215607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547d7c2-789a-4bb2-9693-bbf914e39e4d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:27.304827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 466/955] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:10:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:27.704699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc30d-e91c-4c73-90f9-96351c1e847d None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:27.705202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc30d-e91c-4c73-90f9-96351c1e847d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:27.705202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc30d-e91c-4c73-90f9-96351c1e847d None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:27.706825 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 490/956] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:27.712345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dc58f-5b14-4d85-a044-fa03382718e7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:27.712644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dc58f-5b14-4d85-a044-fa03382718e7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:27.712929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dc58f-5b14-4d85-a044-fa03382718e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:27.732364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-769dc58f-5b14-4d85-a044-fa03382718e7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:27.733930 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-769dc58f-5b14-4d85-a044-fa03382718e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:27.815101 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 467/957] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:27.870470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccc30d-e91c-4c73-90f9-96351c1e847d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:27.969666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccc30d-e91c-4c73-90f9-96351c1e847d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:28.008361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baa7fc-15cd-4692-bced-50a7766c70b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:28.009026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baa7fc-15cd-4692-bced-50a7766c70b3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:28.009334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baa7fc-15cd-4692-bced-50a7766c70b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:28.009640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baa7fc-15cd-4692-bced-50a7766c70b3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:28.010548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7baa7fc-15cd-4692-bced-50a7766c70b3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:28.011967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7baa7fc-15cd-4692-bced-50a7766c70b3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:28.014205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-d7baa7fc-15cd-4692-bced-50a7766c70b3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:28.023362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:28.024683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:28.025638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 491/958] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:28.030925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dc58f-5b14-4d85-a044-fa03382718e7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:28.143844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dc58f-5b14-4d85-a044-fa03382718e7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:28.153788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:28.186380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe7942-92a1-437e-a09a-92dff653e1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:28.187359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe7942-92a1-437e-a09a-92dff653e1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:28.188197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe7942-92a1-437e-a09a-92dff653e1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:28.188473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe7942-92a1-437e-a09a-92dff653e1c1 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:28.189216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe7942-92a1-437e-a09a-92dff653e1c1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:28.190489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe7942-92a1-437e-a09a-92dff653e1c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:28.191491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe7942-92a1-437e-a09a-92dff653e1c1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:28.197704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 468/959] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:28.204109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baa7fc-15cd-4692-bced-50a7766c70b3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:28.308360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baa7fc-15cd-4692-bced-50a7766c70b3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:28.345581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c01a9-7b27-44b1-a16c-44ba8fcb80c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:28.346204 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c01a9-7b27-44b1-a16c-44ba8fcb80c2 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:28.346493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c01a9-7b27-44b1-a16c-44ba8fcb80c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:28.346773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c01a9-7b27-44b1-a16c-44ba8fcb80c2 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:28.347462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c01a9-7b27-44b1-a16c-44ba8fcb80c2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:28.347767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c01a9-7b27-44b1-a16c-44ba8fcb80c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:28.349142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c01a9-7b27-44b1-a16c-44ba8fcb80c2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:28.354311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 492/960] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:28.994475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb660db9-8fd8-4b97-a9a4-f9d6540748c7 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:28.994902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb660db9-8fd8-4b97-a9a4-f9d6540748c7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:28.995087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb660db9-8fd8-4b97-a9a4-f9d6540748c7 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:28.996665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 469/961] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:29.002509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418817f-5b3c-47df-b26d-2ef8e73a614d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.003051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418817f-5b3c-47df-b26d-2ef8e73a614d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.003051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418817f-5b3c-47df-b26d-2ef8e73a614d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.024162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-9418817f-5b3c-47df-b26d-2ef8e73a614d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:29.025436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9418817f-5b3c-47df-b26d-2ef8e73a614d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:29.091889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:29.103428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 493/962] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:29.127479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2cb58-b222-4e0b-8f82-250b37552339 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.128049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2cb58-b222-4e0b-8f82-250b37552339 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.128049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2cb58-b222-4e0b-8f82-250b37552339 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.129539 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 470/963] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:29.129930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418817f-5b3c-47df-b26d-2ef8e73a614d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:29.135570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602c642-e58f-4030-b0a5-688134cf659b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.135866 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602c642-e58f-4030-b0a5-688134cf659b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.136126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602c642-e58f-4030-b0a5-688134cf659b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.155136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0602c642-e58f-4030-b0a5-688134cf659b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:29.156881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0602c642-e58f-4030-b0a5-688134cf659b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:29.236467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 471/964] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:29.258765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418817f-5b3c-47df-b26d-2ef8e73a614d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:29.260499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602c642-e58f-4030-b0a5-688134cf659b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:29.297509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:29.298141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.298418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.298745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.299435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:29.300932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:29.303514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WARNING keystone.server.flask.common [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 06:10:29.340936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.sql.core [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 06:10:29.340936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 06:10:29.340936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [parameters: {'user_id': '966d9d23434940c483cd4353428ff32f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 06:10:29.340936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80317) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application result.read() Jul 28 06:10:29.365224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:10:29.366452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application yield resource Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise self.value Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application session.commit() Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 06:10:29.368322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self.session.flush() Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application compat.raise_( Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 06:10:29.369448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:10:29.370585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application result.read() Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:10:29.379957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [parameters: {'user_id': '966d9d23434940c483cd4353428ff32f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:29.382568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 06:10:29.383804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 06:10:29.384960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 06:10:29.384960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 06:10:29.384960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 06:10:29.384960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 06:10:29.384960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 06:10:29.384960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:29.384960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 494/965] 10.222.0.69 () {60 vars in 1131 bytes} [Thu Jul 28 06:10:29 2022] POST /identity/v3/users => generated 160 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:29.393840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602c642-e58f-4030-b0a5-688134cf659b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:29.393946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:29.394045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:29.414187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9f97e-3ff0-45fa-ae7f-8bba480bffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:29.415079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9f97e-3ff0-45fa-ae7f-8bba480bffd3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.415571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9f97e-3ff0-45fa-ae7f-8bba480bffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.416070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9f97e-3ff0-45fa-ae7f-8bba480bffd3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.428847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9f97e-3ff0-45fa-ae7f-8bba480bffd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-c4c9f97e-3ff0-45fa-ae7f-8bba480bffd3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:29.434135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:29.435840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:29.437555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:29.441935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 472/966] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 28 06:10:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:29.443145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9f97e-3ff0-45fa-ae7f-8bba480bffd3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:29.445795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:29.505334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a88701-7cfc-488d-990c-dc581e13ffaf None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:29.543537 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc25cec-5281-41f2-b8e7-186a437a986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:29.544114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc25cec-5281-41f2-b8e7-186a437a986a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.544401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc25cec-5281-41f2-b8e7-186a437a986a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.544687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc25cec-5281-41f2-b8e7-186a437a986a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.554110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc25cec-5281-41f2-b8e7-186a437a986a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:29.555818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc25cec-5281-41f2-b8e7-186a437a986a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:29.567670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9f97e-3ff0-45fa-ae7f-8bba480bffd3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-edc25cec-5281-41f2-b8e7-186a437a986a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:29.568243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:29.569154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:29.570182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:29.570182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:29.570182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:29.570182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:29.570182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:29.570182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:29.570182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:29.570182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:29.570182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:10:29.570182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:29.574208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 495/967] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:10:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:29.575441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad590394-c357-42ce-bc6f-79d9dfdfe848 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.575733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad590394-c357-42ce-bc6f-79d9dfdfe848 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.576026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad590394-c357-42ce-bc6f-79d9dfdfe848 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.584555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 496/968] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:29.585031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad590394-c357-42ce-bc6f-79d9dfdfe848 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:29.606905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fa56e-38c4-46cf-9715-6baea83dcf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:29.607583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fa56e-38c4-46cf-9715-6baea83dcf79 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.607919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fa56e-38c4-46cf-9715-6baea83dcf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.608208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fa56e-38c4-46cf-9715-6baea83dcf79 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.608909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fa56e-38c4-46cf-9715-6baea83dcf79 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:29.609878 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fa56e-38c4-46cf-9715-6baea83dcf79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:29.611694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fa56e-38c4-46cf-9715-6baea83dcf79 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:29.621546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 473/969] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 28 06:10:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:29.622845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b61bcc-6ee5-4b7f-b7c8-f877318ca984 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.623178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b61bcc-6ee5-4b7f-b7c8-f877318ca984 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.623457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b61bcc-6ee5-4b7f-b7c8-f877318ca984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.643489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-90b61bcc-6ee5-4b7f-b7c8-f877318ca984 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:29.644715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-90b61bcc-6ee5-4b7f-b7c8-f877318ca984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:29.663819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:29.695974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad590394-c357-42ce-bc6f-79d9dfdfe848 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:29.712141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:29.718872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 474/970] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:29.726379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b61bcc-6ee5-4b7f-b7c8-f877318ca984 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:29.733197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15077c0-94a9-41c0-92b0-2388f241ca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:29.733927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15077c0-94a9-41c0-92b0-2388f241ca1e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.734930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15077c0-94a9-41c0-92b0-2388f241ca1e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.735514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15077c0-94a9-41c0-92b0-2388f241ca1e None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.736757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15077c0-94a9-41c0-92b0-2388f241ca1e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:29.737081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15077c0-94a9-41c0-92b0-2388f241ca1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:29.738649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15077c0-94a9-41c0-92b0-2388f241ca1e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:29.780946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 497/971] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 28 06:10:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:29.782727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15077c0-94a9-41c0-92b0-2388f241ca1e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:29.864728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b61bcc-6ee5-4b7f-b7c8-f877318ca984 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:29.899468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15077c0-94a9-41c0-92b0-2388f241ca1e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:29.909650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224ae8c-d47d-4c40-9fb3-18a89eade19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:29.910857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ae8c-d47d-4c40-9fb3-18a89eade19d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.911360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ae8c-d47d-4c40-9fb3-18a89eade19d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.911854 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ae8c-d47d-4c40-9fb3-18a89eade19d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.925134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224ae8c-d47d-4c40-9fb3-18a89eade19d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:29.927158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224ae8c-d47d-4c40-9fb3-18a89eade19d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:29.940404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a326bd-dec0-4768-aed4-10b55eb77a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:29.941069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a326bd-dec0-4768-aed4-10b55eb77a96 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:29.941398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a326bd-dec0-4768-aed4-10b55eb77a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:29.941684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a326bd-dec0-4768-aed4-10b55eb77a96 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:29.942332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a326bd-dec0-4768-aed4-10b55eb77a96 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:29.943726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a326bd-dec0-4768-aed4-10b55eb77a96 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6224ae8c-d47d-4c40-9fb3-18a89eade19d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:29.947193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:29.948246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:29.949269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:29.949269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:29.949269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:29.949269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:29.950799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 475/972] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-66a326bd-dec0-4768-aed4-10b55eb77a96 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:29.956212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:29.957240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:29.958113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 498/973] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:29.960258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224ae8c-d47d-4c40-9fb3-18a89eade19d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:29.965221 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a326bd-dec0-4768-aed4-10b55eb77a96 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:29.971787 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:30.016724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:30.093073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224ae8c-d47d-4c40-9fb3-18a89eade19d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:30.099716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a326bd-dec0-4768-aed4-10b55eb77a96 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:30.130160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48124d25-3508-4d73-91ca-0ea0397c12d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:30.130689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48124d25-3508-4d73-91ca-0ea0397c12d5 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.130980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48124d25-3508-4d73-91ca-0ea0397c12d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.131260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48124d25-3508-4d73-91ca-0ea0397c12d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.131961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48124d25-3508-4d73-91ca-0ea0397c12d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:30.132269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48124d25-3508-4d73-91ca-0ea0397c12d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:30.133731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48124d25-3508-4d73-91ca-0ea0397c12d5 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:30.137823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a041f-6900-4853-b4fa-8aeee03b847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:30.138701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a041f-6900-4853-b4fa-8aeee03b847e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.139190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a041f-6900-4853-b4fa-8aeee03b847e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.139773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a041f-6900-4853-b4fa-8aeee03b847e None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.140788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a041f-6900-4853-b4fa-8aeee03b847e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:30.141465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a041f-6900-4853-b4fa-8aeee03b847e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:30.142924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a041f-6900-4853-b4fa-8aeee03b847e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:30.146470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 476/974] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:30.152099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48124d25-3508-4d73-91ca-0ea0397c12d5 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:30.153366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 499/975] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 28 06:10:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:30.159777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a041f-6900-4853-b4fa-8aeee03b847e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:30.226579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:30.259535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:30.263159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a041f-6900-4853-b4fa-8aeee03b847e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:30.267242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48124d25-3508-4d73-91ca-0ea0397c12d5 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:30.302643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf2acd-cc92-41fb-ad1c-0bf62938ee0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:30.303380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf2acd-cc92-41fb-ad1c-0bf62938ee0c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.303728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf2acd-cc92-41fb-ad1c-0bf62938ee0c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.304076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf2acd-cc92-41fb-ad1c-0bf62938ee0c None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.304827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf2acd-cc92-41fb-ad1c-0bf62938ee0c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:30.305218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf2acd-cc92-41fb-ad1c-0bf62938ee0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:30.305756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36e821-e8a3-489a-9bbc-9461645eb0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:30.306405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36e821-e8a3-489a-9bbc-9461645eb0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.306694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36e821-e8a3-489a-9bbc-9461645eb0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.306999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36e821-e8a3-489a-9bbc-9461645eb0e7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.308491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf2acd-cc92-41fb-ad1c-0bf62938ee0c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:30.314704 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 500/976] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 28 06:10:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:30.318308 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36e821-e8a3-489a-9bbc-9461645eb0e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:30.319889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36e821-e8a3-489a-9bbc-9461645eb0e7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:30.320356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf2acd-cc92-41fb-ad1c-0bf62938ee0c None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-bd36e821-e8a3-489a-9bbc-9461645eb0e7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:30.335077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:30.336299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:30.337273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 477/977] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 28 06:10:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:30.342003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36e821-e8a3-489a-9bbc-9461645eb0e7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:30.436120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf2acd-cc92-41fb-ad1c-0bf62938ee0c None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:30.460237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36e821-e8a3-489a-9bbc-9461645eb0e7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:30.474500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fad40d-848e-4585-8349-b40e80e33980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:30.475132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fad40d-848e-4585-8349-b40e80e33980 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.475423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fad40d-848e-4585-8349-b40e80e33980 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.475782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fad40d-848e-4585-8349-b40e80e33980 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.476562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fad40d-848e-4585-8349-b40e80e33980 None admin] RBAC: Adding query filter params (interface=admin, service_id=cdc63d9cb9794068a2778b41f55ca6db, region_id=RegionOne) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:30.476882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fad40d-848e-4585-8349-b40e80e33980 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:30.478289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fad40d-848e-4585-8349-b40e80e33980 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:30.483500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 501/978] 10.222.0.69 () {56 vars in 1249 bytes} [Thu Jul 28 06:10:30 2022] GET /identity/v3/endpoints?service_id=cdc63d9cb9794068a2778b41f55ca6db&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:30.485495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a896c8a-a543-4701-90f8-858c2a3b0859 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.485620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a896c8a-a543-4701-90f8-858c2a3b0859 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.490960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a896c8a-a543-4701-90f8-858c2a3b0859 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.490960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 502/979] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:30.491579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:30.491702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a896c8a-a543-4701-90f8-858c2a3b0859 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:30.497441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b21c9-1934-4503-9c09-12bff1942475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:30.498074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b21c9-1934-4503-9c09-12bff1942475 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.498360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b21c9-1934-4503-9c09-12bff1942475 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.498674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b21c9-1934-4503-9c09-12bff1942475 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.499513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b21c9-1934-4503-9c09-12bff1942475 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:30.500420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b21c9-1934-4503-9c09-12bff1942475 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:30.502015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b21c9-1934-4503-9c09-12bff1942475 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:30.521405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 478/980] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 28 06:10:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:30.522671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1280-9014-4c48-9927-993b50b00fab None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.522979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1280-9014-4c48-9927-993b50b00fab None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.523239 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1280-9014-4c48-9927-993b50b00fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.525427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:30.541786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b17b1280-9014-4c48-9927-993b50b00fab None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:30.542992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b17b1280-9014-4c48-9927-993b50b00fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:30.600439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a896c8a-a543-4701-90f8-858c2a3b0859 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:30.611771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 479/981] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:30.612614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1280-9014-4c48-9927-993b50b00fab None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:30.635316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521a41a-9129-404b-92ae-98d49a67717e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:30.635951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a41a-9129-404b-92ae-98d49a67717e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.636237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a41a-9129-404b-92ae-98d49a67717e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.636513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a41a-9129-404b-92ae-98d49a67717e None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.637176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521a41a-9129-404b-92ae-98d49a67717e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:30.638583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521a41a-9129-404b-92ae-98d49a67717e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:30.645060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 503/982] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 28 06:10:30 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:10:30.645933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521a41a-9129-404b-92ae-98d49a67717e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:30.739729 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1280-9014-4c48-9927-993b50b00fab None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:30.755575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:30.759441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:30.771384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521a41a-9129-404b-92ae-98d49a67717e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:30.779802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26c0f5-6d02-46b6-88cc-6db1498e5060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:30.780456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26c0f5-6d02-46b6-88cc-6db1498e5060 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.780741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26c0f5-6d02-46b6-88cc-6db1498e5060 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.781022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26c0f5-6d02-46b6-88cc-6db1498e5060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.812994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c2b85-bbf0-48f7-82f6-501854930d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:30.814301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c2b85-bbf0-48f7-82f6-501854930d28 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:30.814301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c2b85-bbf0-48f7-82f6-501854930d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:30.814301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c2b85-bbf0-48f7-82f6-501854930d28 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:30.814874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c2b85-bbf0-48f7-82f6-501854930d28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:30.816817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c2b85-bbf0-48f7-82f6-501854930d28 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:30.860572 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26c0f5-6d02-46b6-88cc-6db1498e5060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:30.863395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26c0f5-6d02-46b6-88cc-6db1498e5060 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:30.893334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 504/983] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:10:30 2022] POST /identity/v3/services => generated 242 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:30.980664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 480/984] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:30 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:10:30.987041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c2b85-bbf0-48f7-82f6-501854930d28 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:31.106669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c2b85-bbf0-48f7-82f6-501854930d28 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:31.143168 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351439d-6e7a-41b4-a97a-b6c63bb54385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:31.143826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351439d-6e7a-41b4-a97a-b6c63bb54385 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:31.144115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351439d-6e7a-41b4-a97a-b6c63bb54385 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:31.144397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351439d-6e7a-41b4-a97a-b6c63bb54385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:31.156734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351439d-6e7a-41b4-a97a-b6c63bb54385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:31.157911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351439d-6e7a-41b4-a97a-b6c63bb54385 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:31.166790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-9351439d-6e7a-41b4-a97a-b6c63bb54385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:31.175159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:31.176142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:31.177037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:31.177037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:31.177037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:31.177037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:31.177037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 505/985] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:31.182909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26c0f5-6d02-46b6-88cc-6db1498e5060 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:31.200875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:31.298838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26c0f5-6d02-46b6-88cc-6db1498e5060 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:31.335599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e821e8-5f64-466d-a8b1-646e876b5205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:31.336226 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e821e8-5f64-466d-a8b1-646e876b5205 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:31.336506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e821e8-5f64-466d-a8b1-646e876b5205 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:31.336780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e821e8-5f64-466d-a8b1-646e876b5205 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:31.337504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e821e8-5f64-466d-a8b1-646e876b5205 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:31.337812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e821e8-5f64-466d-a8b1-646e876b5205 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:31.339189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e821e8-5f64-466d-a8b1-646e876b5205 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:31.359130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 481/986] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:31.361626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351439d-6e7a-41b4-a97a-b6c63bb54385 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:31.475161 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351439d-6e7a-41b4-a97a-b6c63bb54385 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:31.519758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdc93b-d976-4d5d-9b49-1ec2a696f2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:31.520417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdc93b-d976-4d5d-9b49-1ec2a696f2bc None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:31.520706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdc93b-d976-4d5d-9b49-1ec2a696f2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:31.520987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdc93b-d976-4d5d-9b49-1ec2a696f2bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:31.527974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdc93b-d976-4d5d-9b49-1ec2a696f2bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:31.529574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdc93b-d976-4d5d-9b49-1ec2a696f2bc None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-35bdc93b-d976-4d5d-9b49-1ec2a696f2bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:31.552546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:31.554032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:31.555275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:31.555275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:31.555275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:31.555275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:31.555275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:31.555275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:31.555275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:31.555275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:31.556010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 506/987] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:31.559926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e821e8-5f64-466d-a8b1-646e876b5205 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:31.639529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:31.665447 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e821e8-5f64-466d-a8b1-646e876b5205 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:31.700140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388f998-f8ce-4d0e-9061-3e96a9ebf717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:31.700768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388f998-f8ce-4d0e-9061-3e96a9ebf717 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:31.701053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388f998-f8ce-4d0e-9061-3e96a9ebf717 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:31.701358 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388f998-f8ce-4d0e-9061-3e96a9ebf717 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:31.702130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388f998-f8ce-4d0e-9061-3e96a9ebf717 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:31.702436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388f998-f8ce-4d0e-9061-3e96a9ebf717 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:31.703950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388f998-f8ce-4d0e-9061-3e96a9ebf717 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:31.716363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 482/988] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:31.722808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdc93b-d976-4d5d-9b49-1ec2a696f2bc None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:31.786553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:31.832721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdc93b-d976-4d5d-9b49-1ec2a696f2bc None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:31.868822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24e8f3-85ae-4a81-b651-be3bb0340ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:31.869473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e8f3-85ae-4a81-b651-be3bb0340ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:31.869768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e8f3-85ae-4a81-b651-be3bb0340ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:31.870046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e8f3-85ae-4a81-b651-be3bb0340ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:31.870840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24e8f3-85ae-4a81-b651-be3bb0340ff8 None admin] RBAC: Adding query filter params (role.id=7945ffd9805a4831932cab7339f7993c, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=39c6f0ee9b78413eb8c5efc7adf145b6) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:31.871157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24e8f3-85ae-4a81-b651-be3bb0340ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:31.872599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24e8f3-85ae-4a81-b651-be3bb0340ff8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:31.880450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 507/989] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:31 2022] GET /identity/v3/role_assignments?user.id=39c6f0ee9b78413eb8c5efc7adf145b6&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=7945ffd9805a4831932cab7339f7993c => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:32.740918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b9dd-ee34-4ada-8d3f-be2123cd86fb None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:32.741525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b9dd-ee34-4ada-8d3f-be2123cd86fb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:32.741525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b9dd-ee34-4ada-8d3f-be2123cd86fb None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:32.742863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 483/990] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:32.748555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a6379-2292-47db-b340-749d6b86d2c3 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:32.749110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a6379-2292-47db-b340-749d6b86d2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:32.749110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a6379-2292-47db-b340-749d6b86d2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:32.768191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-508a6379-2292-47db-b340-749d6b86d2c3 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:32.769749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-508a6379-2292-47db-b340-749d6b86d2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:32.837686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 508/991] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:32.861266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64b9dd-ee34-4ada-8d3f-be2123cd86fb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:32.945289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cabdf-d6fa-48e4-8794-6512c3b96e27 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:32.945605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cabdf-d6fa-48e4-8794-6512c3b96e27 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:32.945888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cabdf-d6fa-48e4-8794-6512c3b96e27 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:32.947378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 509/992] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:10:32.951971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abcbdf-294d-49f7-80ce-e434b8696308 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:32.952257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abcbdf-294d-49f7-80ce-e434b8696308 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:32.952536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abcbdf-294d-49f7-80ce-e434b8696308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:32.969464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64b9dd-ee34-4ada-8d3f-be2123cd86fb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:32.973802 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-36abcbdf-294d-49f7-80ce-e434b8696308 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:32.975009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-36abcbdf-294d-49f7-80ce-e434b8696308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:33.007565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a9f6-e2de-4da2-a771-edcaef881162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:33.008226 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a9f6-e2de-4da2-a771-edcaef881162 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.008513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a9f6-e2de-4da2-a771-edcaef881162 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.008799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a9f6-e2de-4da2-a771-edcaef881162 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.009476 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01a9f6-e2de-4da2-a771-edcaef881162 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:33.011135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01a9f6-e2de-4da2-a771-edcaef881162 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:33.012992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-2a01a9f6-e2de-4da2-a771-edcaef881162 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:33.021633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:33.022725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:33.023674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 484/993] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 28 06:10:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:33.029474 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a9f6-e2de-4da2-a771-edcaef881162 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:33.035477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:33.054097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 510/994] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:33.077612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abcbdf-294d-49f7-80ce-e434b8696308 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:33.143773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a9f6-e2de-4da2-a771-edcaef881162 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:33.193273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4807979-6c2c-4a92-9a89-ddda5512101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:33.193920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4807979-6c2c-4a92-9a89-ddda5512101b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.194214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4807979-6c2c-4a92-9a89-ddda5512101b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.194473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4807979-6c2c-4a92-9a89-ddda5512101b None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.195270 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4807979-6c2c-4a92-9a89-ddda5512101b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:33.195582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4807979-6c2c-4a92-9a89-ddda5512101b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:33.197081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4807979-6c2c-4a92-9a89-ddda5512101b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:33.203581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abcbdf-294d-49f7-80ce-e434b8696308 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:33.207438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 485/995] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 28 06:10:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:33.208175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4807979-6c2c-4a92-9a89-ddda5512101b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:33.241521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707232d-5293-4bfc-8320-77ce1c547e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:33.242174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707232d-5293-4bfc-8320-77ce1c547e66 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.242456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707232d-5293-4bfc-8320-77ce1c547e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.242752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707232d-5293-4bfc-8320-77ce1c547e66 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.243372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707232d-5293-4bfc-8320-77ce1c547e66 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:33.244780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707232d-5293-4bfc-8320-77ce1c547e66 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-e707232d-5293-4bfc-8320-77ce1c547e66 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:33.254576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:33.255872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:33.256754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 511/996] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:33.261918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707232d-5293-4bfc-8320-77ce1c547e66 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:33.283215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:33.317990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:33.331109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4807979-6c2c-4a92-9a89-ddda5512101b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:33.373867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27aea26-e4e2-4b4b-a5c0-cd0ee99c5762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:33.374499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27aea26-e4e2-4b4b-a5c0-cd0ee99c5762 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.374834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27aea26-e4e2-4b4b-a5c0-cd0ee99c5762 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.375117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27aea26-e4e2-4b4b-a5c0-cd0ee99c5762 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.375808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27aea26-e4e2-4b4b-a5c0-cd0ee99c5762 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:33.376119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27aea26-e4e2-4b4b-a5c0-cd0ee99c5762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:33.377528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27aea26-e4e2-4b4b-a5c0-cd0ee99c5762 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:33.379078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707232d-5293-4bfc-8320-77ce1c547e66 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:33.383736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 486/997] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 28 06:10:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:33.389902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27aea26-e4e2-4b4b-a5c0-cd0ee99c5762 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:33.422671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782ea04-9a44-45ef-b792-f18a34ca984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:33.423304 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782ea04-9a44-45ef-b792-f18a34ca984b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.423592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782ea04-9a44-45ef-b792-f18a34ca984b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.423876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782ea04-9a44-45ef-b792-f18a34ca984b None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.424554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782ea04-9a44-45ef-b792-f18a34ca984b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:33.424863 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782ea04-9a44-45ef-b792-f18a34ca984b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:33.426283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782ea04-9a44-45ef-b792-f18a34ca984b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:33.431692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 512/998] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:33.437384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782ea04-9a44-45ef-b792-f18a34ca984b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:33.497982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27aea26-e4e2-4b4b-a5c0-cd0ee99c5762 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:33.528537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:33.540036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d166b-9dda-495e-890b-831cdc42352d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:33.540664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d166b-9dda-495e-890b-831cdc42352d None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.540957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d166b-9dda-495e-890b-831cdc42352d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.541242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d166b-9dda-495e-890b-831cdc42352d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.541998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d166b-9dda-495e-890b-831cdc42352d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:33.543433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d166b-9dda-495e-890b-831cdc42352d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:33.547081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782ea04-9a44-45ef-b792-f18a34ca984b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:33.561008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:33.582678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ad5f0-ae6d-4097-b938-881177c2f9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:33.583303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ad5f0-ae6d-4097-b938-881177c2f9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.583590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ad5f0-ae6d-4097-b938-881177c2f9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.583870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ad5f0-ae6d-4097-b938-881177c2f9c9 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.584554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ad5f0-ae6d-4097-b938-881177c2f9c9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:33.584863 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ad5f0-ae6d-4097-b938-881177c2f9c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:33.586287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ad5f0-ae6d-4097-b938-881177c2f9c9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:33.591069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 513/999] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:33.597852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ad5f0-ae6d-4097-b938-881177c2f9c9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:33.651691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 487/1000] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:10:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:10:33.730603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ad5f0-ae6d-4097-b938-881177c2f9c9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:33.775645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac2e95-a56d-4bea-a7b6-67a8e4173847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:33.776408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac2e95-a56d-4bea-a7b6-67a8e4173847 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.776820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac2e95-a56d-4bea-a7b6-67a8e4173847 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.776820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac2e95-a56d-4bea-a7b6-67a8e4173847 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.777635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac2e95-a56d-4bea-a7b6-67a8e4173847 None admin] RBAC: Adding query filter params (interface=public, service_id=d5b436392b89405b9eb579888fbf1361, region_id=RegionOne) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:33.777955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac2e95-a56d-4bea-a7b6-67a8e4173847 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:33.779431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac2e95-a56d-4bea-a7b6-67a8e4173847 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:33.790638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 514/1001] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 28 06:10:33 2022] GET /identity/v3/endpoints?service_id=d5b436392b89405b9eb579888fbf1361&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:33.791533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d166b-9dda-495e-890b-831cdc42352d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:33.905014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d166b-9dda-495e-890b-831cdc42352d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:33.932241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8bf53-8fbd-4afb-8320-e99fce8d5aff None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.932749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8bf53-8fbd-4afb-8320-e99fce8d5aff None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.935010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8bf53-8fbd-4afb-8320-e99fce8d5aff None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.935361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 515/1002] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:33.942202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695491dc-d09e-499e-b785-09aeb86549c2 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.943078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695491dc-d09e-499e-b785-09aeb86549c2 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.943078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695491dc-d09e-499e-b785-09aeb86549c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.949506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5d951-e560-40cc-bff7-f001982704fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:33.950255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5d951-e560-40cc-bff7-f001982704fd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:33.950470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5d951-e560-40cc-bff7-f001982704fd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:33.950687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5d951-e560-40cc-bff7-f001982704fd None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:33.951355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5d951-e560-40cc-bff7-f001982704fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:33.953754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5d951-e560-40cc-bff7-f001982704fd None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:33.969339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 488/1003] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 28 06:10:33 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:10:33.969561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-695491dc-d09e-499e-b785-09aeb86549c2 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:33.969561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-695491dc-d09e-499e-b785-09aeb86549c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:34.008223 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:34.053795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 516/1004] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:33 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:34.076051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5d951-e560-40cc-bff7-f001982704fd None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:34.133150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:34.186991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5d951-e560-40cc-bff7-f001982704fd None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:34.222591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99211df3-a110-43b9-8010-f04d83965403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:34.223246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99211df3-a110-43b9-8010-f04d83965403 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:34.223536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99211df3-a110-43b9-8010-f04d83965403 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:34.223812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99211df3-a110-43b9-8010-f04d83965403 None admin] PATH_INFO: `/v3/services/image` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:34.224545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99211df3-a110-43b9-8010-f04d83965403 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:34.225964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99211df3-a110-43b9-8010-f04d83965403 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-99211df3-a110-43b9-8010-f04d83965403 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:34.235788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:34.236884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:34.237801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 489/1005] 10.222.0.69 () {56 vars in 1105 bytes} [Thu Jul 28 06:10:34 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:34.244675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695491dc-d09e-499e-b785-09aeb86549c2 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:34.358010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695491dc-d09e-499e-b785-09aeb86549c2 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:34.400552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e8dbc-1246-44c8-a04b-7e2686af0f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:34.401082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e8dbc-1246-44c8-a04b-7e2686af0f4c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:34.401398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e8dbc-1246-44c8-a04b-7e2686af0f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:34.401681 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e8dbc-1246-44c8-a04b-7e2686af0f4c None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:34.402527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e8dbc-1246-44c8-a04b-7e2686af0f4c None admin] RBAC: Adding query filter params (name=image) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:34.402843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e8dbc-1246-44c8-a04b-7e2686af0f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:34.404498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e8dbc-1246-44c8-a04b-7e2686af0f4c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:34.415684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 517/1006] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:34.416988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99211df3-a110-43b9-8010-f04d83965403 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:34.530058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99211df3-a110-43b9-8010-f04d83965403 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:34.544637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:34.565731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c362ee4-8be5-4140-a89e-0197bd0ac659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:34.566355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c362ee4-8be5-4140-a89e-0197bd0ac659 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:34.566642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c362ee4-8be5-4140-a89e-0197bd0ac659 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:34.566923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c362ee4-8be5-4140-a89e-0197bd0ac659 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:34.567611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c362ee4-8be5-4140-a89e-0197bd0ac659 None admin] RBAC: Adding query filter params (type=image) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:34.567919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c362ee4-8be5-4140-a89e-0197bd0ac659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:34.569335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c362ee4-8be5-4140-a89e-0197bd0ac659 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:34.573715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 490/1007] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:35.725940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afe5b6-5224-4a92-81e8-9f65f07e22b5 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:35.726465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afe5b6-5224-4a92-81e8-9f65f07e22b5 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:35.726465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afe5b6-5224-4a92-81e8-9f65f07e22b5 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:35.727896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 518/1008] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:35.733034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353998e-228d-4127-8831-81dc3194aac7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:35.733488 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353998e-228d-4127-8831-81dc3194aac7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:35.733630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353998e-228d-4127-8831-81dc3194aac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:35.753274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7353998e-228d-4127-8831-81dc3194aac7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:35.754490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7353998e-228d-4127-8831-81dc3194aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:35.825084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 491/1009] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:35 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:35.847851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afe5b6-5224-4a92-81e8-9f65f07e22b5 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:35.880656 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:35.956266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afe5b6-5224-4a92-81e8-9f65f07e22b5 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:35.993612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33675f52-b4b8-470c-b0f3-79e7605dcb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:35.994200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33675f52-b4b8-470c-b0f3-79e7605dcb0d None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:35.994500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33675f52-b4b8-470c-b0f3-79e7605dcb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:35.994786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33675f52-b4b8-470c-b0f3-79e7605dcb0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:35.995972 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33675f52-b4b8-470c-b0f3-79e7605dcb0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:35.997571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33675f52-b4b8-470c-b0f3-79e7605dcb0d None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:36.004220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 519/1010] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:36.005216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353998e-228d-4127-8831-81dc3194aac7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:36.031257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ed7a5-2e31-4fa1-a475-eb602b44cd05 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:36.031567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ed7a5-2e31-4fa1-a475-eb602b44cd05 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:36.031819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ed7a5-2e31-4fa1-a475-eb602b44cd05 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:36.033215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 520/1011] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:10:36.038130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf8407-60ec-4452-941e-2d46dfb3331f None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:36.038413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf8407-60ec-4452-941e-2d46dfb3331f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:36.038697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf8407-60ec-4452-941e-2d46dfb3331f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:36.057337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-3adf8407-60ec-4452-941e-2d46dfb3331f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:36.058547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3adf8407-60ec-4452-941e-2d46dfb3331f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:36.119319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353998e-228d-4127-8831-81dc3194aac7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:36.130326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:36.131003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 521/1012] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:36.154100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf8407-60ec-4452-941e-2d46dfb3331f None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:36.157882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7937b9-2bac-4dd3-9abf-528a9757c04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:36.158759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7937b9-2bac-4dd3-9abf-528a9757c04f None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:36.159238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7937b9-2bac-4dd3-9abf-528a9757c04f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:36.159570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7937b9-2bac-4dd3-9abf-528a9757c04f None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:36.160214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7937b9-2bac-4dd3-9abf-528a9757c04f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:36.161701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7937b9-2bac-4dd3-9abf-528a9757c04f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:36.184562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:36.244898 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 492/1013] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:10:36 2022] POST /identity/v3/projects => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:36.259454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf8407-60ec-4452-941e-2d46dfb3331f None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:36.303357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b77be3-a297-46ae-8184-a1c486911d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:36.304034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77be3-a297-46ae-8184-a1c486911d5a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:36.304452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77be3-a297-46ae-8184-a1c486911d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:36.304623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77be3-a297-46ae-8184-a1c486911d5a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:36.305245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b77be3-a297-46ae-8184-a1c486911d5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:36.306685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b77be3-a297-46ae-8184-a1c486911d5a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-81b77be3-a297-46ae-8184-a1c486911d5a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:36.318349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:36.319421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:36.325647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 522/1014] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:36.328741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7937b9-2bac-4dd3-9abf-528a9757c04f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:36.459060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7937b9-2bac-4dd3-9abf-528a9757c04f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:36.497438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebc0b1-f4c1-40ea-9158-f4f5617e9495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:36.498078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebc0b1-f4c1-40ea-9158-f4f5617e9495 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:36.498366 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebc0b1-f4c1-40ea-9158-f4f5617e9495 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:36.498648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebc0b1-f4c1-40ea-9158-f4f5617e9495 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:36.499368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebc0b1-f4c1-40ea-9158-f4f5617e9495 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:36.499704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebc0b1-f4c1-40ea-9158-f4f5617e9495 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:36.501160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebc0b1-f4c1-40ea-9158-f4f5617e9495 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:36.507560 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 493/1015] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:36.512837 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b77be3-a297-46ae-8184-a1c486911d5a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:36.616185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf04bf-f557-4556-bb40-80314f6285dd None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:36.616542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf04bf-f557-4556-bb40-80314f6285dd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:36.616749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf04bf-f557-4556-bb40-80314f6285dd None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:36.618407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 494/1016] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:36.619052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:36.623412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720aab8-07d9-4e4d-8ef4-d47d28ca5121 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:36.623698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720aab8-07d9-4e4d-8ef4-d47d28ca5121 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:36.623971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720aab8-07d9-4e4d-8ef4-d47d28ca5121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:36.629109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b77be3-a297-46ae-8184-a1c486911d5a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:36.643152 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b720aab8-07d9-4e4d-8ef4-d47d28ca5121 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:36.644368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b720aab8-07d9-4e4d-8ef4-d47d28ca5121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:36.664023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:36.666474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1a92b-8a4f-4ffd-832c-edab0ea6546f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:36.667105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1a92b-8a4f-4ffd-832c-edab0ea6546f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:36.667398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1a92b-8a4f-4ffd-832c-edab0ea6546f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:36.667673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1a92b-8a4f-4ffd-832c-edab0ea6546f None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:36.668360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1a92b-8a4f-4ffd-832c-edab0ea6546f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:36.668666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1a92b-8a4f-4ffd-832c-edab0ea6546f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:36.670157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1a92b-8a4f-4ffd-832c-edab0ea6546f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:36.675872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 523/1017] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 28 06:10:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:36.681430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1a92b-8a4f-4ffd-832c-edab0ea6546f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:36.716357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 495/1018] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:36.738702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720aab8-07d9-4e4d-8ef4-d47d28ca5121 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:36.801247 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1a92b-8a4f-4ffd-832c-edab0ea6546f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:36.839925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe3c35-b129-4588-9cab-c99bf35148f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:36.840941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe3c35-b129-4588-9cab-c99bf35148f4 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:36.841485 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe3c35-b129-4588-9cab-c99bf35148f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:36.842398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe3c35-b129-4588-9cab-c99bf35148f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:36.843869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe3c35-b129-4588-9cab-c99bf35148f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:36.845496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe3c35-b129-4588-9cab-c99bf35148f4 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:36.855608 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720aab8-07d9-4e4d-8ef4-d47d28ca5121 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:36.891596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be15f11-27b6-477f-b677-2f8468865a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:36.892214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be15f11-27b6-477f-b677-2f8468865a21 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:36.892501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be15f11-27b6-477f-b677-2f8468865a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:36.892778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be15f11-27b6-477f-b677-2f8468865a21 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:36.893509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be15f11-27b6-477f-b677-2f8468865a21 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:36.894920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be15f11-27b6-477f-b677-2f8468865a21 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:36.925977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 524/1019] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:10:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:10:36.997331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 496/1020] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:10:36 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:10:38.364576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf56f4f-92fd-4512-997c-cdae4c81b37d None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:38.365131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf56f4f-92fd-4512-997c-cdae4c81b37d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:38.365131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf56f4f-92fd-4512-997c-cdae4c81b37d None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:38.366809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 525/1021] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:38.372749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a837c2-4e68-445b-bec9-262dbcb3f305 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:38.373172 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a837c2-4e68-445b-bec9-262dbcb3f305 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:38.373389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a837c2-4e68-445b-bec9-262dbcb3f305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:38.393574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-39a837c2-4e68-445b-bec9-262dbcb3f305 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:38.394781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-39a837c2-4e68-445b-bec9-262dbcb3f305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:38.405218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:38.472991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 497/1022] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:38.498534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf56f4f-92fd-4512-997c-cdae4c81b37d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:38.517057 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:38.607879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf56f4f-92fd-4512-997c-cdae4c81b37d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:38.643910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ce341-0f71-4f34-8440-95c1ccb15be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:38.644479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ce341-0f71-4f34-8440-95c1ccb15be7 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:38.644768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ce341-0f71-4f34-8440-95c1ccb15be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:38.645051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ce341-0f71-4f34-8440-95c1ccb15be7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:38.646258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ce341-0f71-4f34-8440-95c1ccb15be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:38.647708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ce341-0f71-4f34-8440-95c1ccb15be7 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:38.653994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 526/1023] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:38.655264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a837c2-4e68-445b-bec9-262dbcb3f305 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:38.761430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a837c2-4e68-445b-bec9-262dbcb3f305 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:38.802736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fd87f-9187-4e1c-803d-ad33fcd55139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:38.803371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fd87f-9187-4e1c-803d-ad33fcd55139 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:38.803655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fd87f-9187-4e1c-803d-ad33fcd55139 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:38.803934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fd87f-9187-4e1c-803d-ad33fcd55139 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:38.804628 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fd87f-9187-4e1c-803d-ad33fcd55139 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:38.806101 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fd87f-9187-4e1c-803d-ad33fcd55139 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:38.818846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:38.957554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 498/1024] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:10:38 2022] POST /identity/v3/users => generated 285 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:39.063483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4070fed-0cdf-4053-92b7-df32b7243581 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.063865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4070fed-0cdf-4053-92b7-df32b7243581 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.063970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b381b2-1099-4eca-a83b-d2377041426a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.064052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4070fed-0cdf-4053-92b7-df32b7243581 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.064255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b381b2-1099-4eca-a83b-d2377041426a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.064534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b381b2-1099-4eca-a83b-d2377041426a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.065691 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 527/1025] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:39.066150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 499/1026] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:10:39.070990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9e119-2d5b-4083-be9f-ab2d2122226c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.071283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9e119-2d5b-4083-be9f-ab2d2122226c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.071565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9e119-2d5b-4083-be9f-ab2d2122226c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.074833 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849238e-7f05-4291-a6f2-17cb284e7b3d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.075124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849238e-7f05-4291-a6f2-17cb284e7b3d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.075404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849238e-7f05-4291-a6f2-17cb284e7b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.090393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-beb9e119-2d5b-4083-be9f-ab2d2122226c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:39.091622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-beb9e119-2d5b-4083-be9f-ab2d2122226c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:39.094461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1849238e-7f05-4291-a6f2-17cb284e7b3d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:39.095668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1849238e-7f05-4291-a6f2-17cb284e7b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:39.174756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 500/1027] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:39.175430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 528/1028] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:39.198084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849238e-7f05-4291-a6f2-17cb284e7b3d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:39.199995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9e119-2d5b-4083-be9f-ab2d2122226c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:39.213211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:39.307950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9e119-2d5b-4083-be9f-ab2d2122226c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:39.314657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849238e-7f05-4291-a6f2-17cb284e7b3d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:39.341457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:39.349354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf20eba-5254-427f-beba-31fceccbb548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:39.350233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf20eba-5254-427f-beba-31fceccbb548 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.350233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf20eba-5254-427f-beba-31fceccbb548 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.350562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf20eba-5254-427f-beba-31fceccbb548 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.354150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc0b6-9f59-42f7-b532-65cb00055b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:39.354802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc0b6-9f59-42f7-b532-65cb00055b8f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.359542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc0b6-9f59-42f7-b532-65cb00055b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.359542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc0b6-9f59-42f7-b532-65cb00055b8f None admin] PATH_INFO: `/v3/services/image` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.359542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfc0b6-9f59-42f7-b532-65cb00055b8f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:39.359542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfc0b6-9f59-42f7-b532-65cb00055b8f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:39.366805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf20eba-5254-427f-beba-31fceccbb548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-1bf20eba-5254-427f-beba-31fceccbb548 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:39.371678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:39.372626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:39.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-02cfc0b6-9f59-42f7-b532-65cb00055b8f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:39.380117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:39.381384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:10:39.382255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:39.383017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 529/1029] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 28 06:10:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:39.383087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 501/1030] 10.222.0.69 () {56 vars in 1105 bytes} [Thu Jul 28 06:10:39 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:39.383153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf20eba-5254-427f-beba-31fceccbb548 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:39.385364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc0b6-9f59-42f7-b532-65cb00055b8f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:39.459095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:39.492039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf20eba-5254-427f-beba-31fceccbb548 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:39.498510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc0b6-9f59-42f7-b532-65cb00055b8f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:39.529904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90aa60-07cd-48fe-bc3a-387f69e28bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:39.530721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90aa60-07cd-48fe-bc3a-387f69e28bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.531027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90aa60-07cd-48fe-bc3a-387f69e28bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.531311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90aa60-07cd-48fe-bc3a-387f69e28bb1 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.532005 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90aa60-07cd-48fe-bc3a-387f69e28bb1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:39.532312 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90aa60-07cd-48fe-bc3a-387f69e28bb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:39.533722 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90aa60-07cd-48fe-bc3a-387f69e28bb1 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:39.536905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6a3e9-88ce-46f9-ae07-414d9f965769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:39.537798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6a3e9-88ce-46f9-ae07-414d9f965769 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.538273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6a3e9-88ce-46f9-ae07-414d9f965769 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.538738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6a3e9-88ce-46f9-ae07-414d9f965769 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.539041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 530/1031] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:39.539836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6a3e9-88ce-46f9-ae07-414d9f965769 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:39.540151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6a3e9-88ce-46f9-ae07-414d9f965769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:39.541593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6a3e9-88ce-46f9-ae07-414d9f965769 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:39.544518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90aa60-07cd-48fe-bc3a-387f69e28bb1 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:39.553820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 502/1032] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 28 06:10:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:39.559508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6a3e9-88ce-46f9-ae07-414d9f965769 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:39.619013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:39.653557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90aa60-07cd-48fe-bc3a-387f69e28bb1 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:39.677076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6a3e9-88ce-46f9-ae07-414d9f965769 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:39.692675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b4b12-23c8-4af1-b34e-7b67281f9b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:39.693337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b4b12-23c8-4af1-b34e-7b67281f9b7b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.693625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b4b12-23c8-4af1-b34e-7b67281f9b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.694103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b4b12-23c8-4af1-b34e-7b67281f9b7b None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.695189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b4b12-23c8-4af1-b34e-7b67281f9b7b None admin] RBAC: Adding query filter params (type=image) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:39.695532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b4b12-23c8-4af1-b34e-7b67281f9b7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:39.697389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b4b12-23c8-4af1-b34e-7b67281f9b7b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:39.699059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:39.709155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 531/1033] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:39.710368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b4b12-23c8-4af1-b34e-7b67281f9b7b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:39.716300 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd421c76-2f73-4d29-9e59-86f4962d0f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:39.716955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd421c76-2f73-4d29-9e59-86f4962d0f60 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.717240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd421c76-2f73-4d29-9e59-86f4962d0f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.717555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd421c76-2f73-4d29-9e59-86f4962d0f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.728748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd421c76-2f73-4d29-9e59-86f4962d0f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:39.730261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd421c76-2f73-4d29-9e59-86f4962d0f60 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-bd421c76-2f73-4d29-9e59-86f4962d0f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:39.745537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:39.746643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:39.747573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:39.747573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:39.747573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:39.747573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:39.747573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 503/1034] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:39.752741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd421c76-2f73-4d29-9e59-86f4962d0f60 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:39.832113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b4b12-23c8-4af1-b34e-7b67281f9b7b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:39.862936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd421c76-2f73-4d29-9e59-86f4962d0f60 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:39.866200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:39.868790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6d636-7d7d-4da5-b088-a13a90d12972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:39.869607 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6d636-7d7d-4da5-b088-a13a90d12972 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.869816 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6d636-7d7d-4da5-b088-a13a90d12972 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.870215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6d636-7d7d-4da5-b088-a13a90d12972 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.871050 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6d636-7d7d-4da5-b088-a13a90d12972 None admin] RBAC: Adding query filter params (interface=public, service_id=cd8a9e2327e843f4b46ba4d64b953360, region_id=RegionOne) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:39.871427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6d636-7d7d-4da5-b088-a13a90d12972 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:39.872862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6d636-7d7d-4da5-b088-a13a90d12972 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:39.879245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 532/1035] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 28 06:10:39 2022] GET /identity/v3/endpoints?service_id=cd8a9e2327e843f4b46ba4d64b953360&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:39.884910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6d636-7d7d-4da5-b088-a13a90d12972 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:39.902303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc9e9-b427-4e79-a602-1662cfa463ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:39.903169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc9e9-b427-4e79-a602-1662cfa463ae None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:39.903597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc9e9-b427-4e79-a602-1662cfa463ae None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:39.904020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc9e9-b427-4e79-a602-1662cfa463ae None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:39.904869 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc9e9-b427-4e79-a602-1662cfa463ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:39.905339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc9e9-b427-4e79-a602-1662cfa463ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:39.907814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc9e9-b427-4e79-a602-1662cfa463ae None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:39.926482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 504/1036] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:39.927524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc9e9-b427-4e79-a602-1662cfa463ae None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:39.951233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:39.993703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6d636-7d7d-4da5-b088-a13a90d12972 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:40.031854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15f265-3c95-4152-9d5c-577776898cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:40.032477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f265-3c95-4152-9d5c-577776898cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:40.032761 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f265-3c95-4152-9d5c-577776898cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:40.032867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc9e9-b427-4e79-a602-1662cfa463ae None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:40.033060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f265-3c95-4152-9d5c-577776898cd9 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:40.033705 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15f265-3c95-4152-9d5c-577776898cd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:40.035278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15f265-3c95-4152-9d5c-577776898cd9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:40.042191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 533/1037] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 28 06:10:39 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:10:40.070136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011fc45-ffe4-48fc-af20-dd57f645eb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:40.070813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011fc45-ffe4-48fc-af20-dd57f645eb1e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:40.071097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011fc45-ffe4-48fc-af20-dd57f645eb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:40.071374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011fc45-ffe4-48fc-af20-dd57f645eb1e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:40.080263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011fc45-ffe4-48fc-af20-dd57f645eb1e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:40.081796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011fc45-ffe4-48fc-af20-dd57f645eb1e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4011fc45-ffe4-48fc-af20-dd57f645eb1e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:40.095390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:40.096531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:40.097977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:40.097977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:40.097977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:40.097977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:40.097977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:40.097977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:40.097977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:40.097977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:40.097977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:10:40.097977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:40.101744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 505/1038] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 28 06:10:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:40.102823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15f265-3c95-4152-9d5c-577776898cd9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:40.226961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15f265-3c95-4152-9d5c-577776898cd9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:40.254039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:40.262729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317caa06-1c21-4cae-9847-db65f3ec9466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:40.263357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317caa06-1c21-4cae-9847-db65f3ec9466 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:40.263645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317caa06-1c21-4cae-9847-db65f3ec9466 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:40.268651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317caa06-1c21-4cae-9847-db65f3ec9466 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:40.268651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317caa06-1c21-4cae-9847-db65f3ec9466 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:40.268651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317caa06-1c21-4cae-9847-db65f3ec9466 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:40.268651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317caa06-1c21-4cae-9847-db65f3ec9466 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:40.286047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 534/1039] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 28 06:10:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:40.291537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011fc45-ffe4-48fc-af20-dd57f645eb1e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:40.320331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:40.413279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011fc45-ffe4-48fc-af20-dd57f645eb1e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:40.447897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b137c99-825d-4b59-a860-a940264bfa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:40.448516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b137c99-825d-4b59-a860-a940264bfa5e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:40.448804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b137c99-825d-4b59-a860-a940264bfa5e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:40.449082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b137c99-825d-4b59-a860-a940264bfa5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:40.520922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b137c99-825d-4b59-a860-a940264bfa5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:40.523367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b137c99-825d-4b59-a860-a940264bfa5e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:40.548593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:40.626530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 506/1040] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:40 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:10:40.631979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317caa06-1c21-4cae-9847-db65f3ec9466 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:40.739331 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317caa06-1c21-4cae-9847-db65f3ec9466 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:40.779096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f576d4-829c-48f1-bb60-9eb4da8b25fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:40.779757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f576d4-829c-48f1-bb60-9eb4da8b25fe None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:40.780044 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f576d4-829c-48f1-bb60-9eb4da8b25fe None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:40.780496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f576d4-829c-48f1-bb60-9eb4da8b25fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:40.791498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f576d4-829c-48f1-bb60-9eb4da8b25fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:40.792950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f576d4-829c-48f1-bb60-9eb4da8b25fe None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-28f576d4-829c-48f1-bb60-9eb4da8b25fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:40.808356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:40.814812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:40.815796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:40.815796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:40.815796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:40.815796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:40.815796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 535/1041] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:40.816092 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b137c99-825d-4b59-a860-a940264bfa5e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:40.918582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b137c99-825d-4b59-a860-a940264bfa5e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:40.944008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:40.955112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1d46d-4bc5-47cd-8e8a-94b77b2f69bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:40.963475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1d46d-4bc5-47cd-8e8a-94b77b2f69bb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:40.963475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1d46d-4bc5-47cd-8e8a-94b77b2f69bb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:40.963475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1d46d-4bc5-47cd-8e8a-94b77b2f69bb None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:40.963475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1d46d-4bc5-47cd-8e8a-94b77b2f69bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:40.963475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1d46d-4bc5-47cd-8e8a-94b77b2f69bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:40.963475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1d46d-4bc5-47cd-8e8a-94b77b2f69bb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:40.974340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 507/1042] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:40.975763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f576d4-829c-48f1-bb60-9eb4da8b25fe None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:41.014564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:41.081343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f576d4-829c-48f1-bb60-9eb4da8b25fe None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:41.082166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0fde-7247-438c-b3bf-3ad694c57572 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:41.082462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0fde-7247-438c-b3bf-3ad694c57572 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:41.082755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0fde-7247-438c-b3bf-3ad694c57572 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:41.084128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 508/1043] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:41.089023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5f453-4cb1-484b-ba1a-ca343c4cad0d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:41.089329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5f453-4cb1-484b-ba1a-ca343c4cad0d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:41.089645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5f453-4cb1-484b-ba1a-ca343c4cad0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:41.108270 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-efe5f453-4cb1-484b-ba1a-ca343c4cad0d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:41.110193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-efe5f453-4cb1-484b-ba1a-ca343c4cad0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:41.118535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2498011-1901-4794-9fe4-9fd65fca7759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:41.119132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498011-1901-4794-9fe4-9fd65fca7759 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:41.119416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498011-1901-4794-9fe4-9fd65fca7759 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:41.119694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498011-1901-4794-9fe4-9fd65fca7759 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:41.130358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2498011-1901-4794-9fe4-9fd65fca7759 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:41.131684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2498011-1901-4794-9fe4-9fd65fca7759 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-d2498011-1901-4794-9fe4-9fd65fca7759 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:41.144525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:41.145666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:41.151194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:41.151194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:41.151194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:41.151194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:41.151194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:41.151194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:41.151194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:41.151194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:41.151194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 536/1044] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:41.151679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2498011-1901-4794-9fe4-9fd65fca7759 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:41.189391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 509/1045] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:41.213112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5f453-4cb1-484b-ba1a-ca343c4cad0d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:41.260226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:41.267301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2498011-1901-4794-9fe4-9fd65fca7759 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:41.307125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d2f2f-1594-47ee-95ef-51fe5c1ac19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:41.307758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d2f2f-1594-47ee-95ef-51fe5c1ac19a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:41.308090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d2f2f-1594-47ee-95ef-51fe5c1ac19a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:41.308386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d2f2f-1594-47ee-95ef-51fe5c1ac19a None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:41.309190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d2f2f-1594-47ee-95ef-51fe5c1ac19a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:41.309537 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d2f2f-1594-47ee-95ef-51fe5c1ac19a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:41.310981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d2f2f-1594-47ee-95ef-51fe5c1ac19a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:41.324322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 537/1046] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:41.327861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5f453-4cb1-484b-ba1a-ca343c4cad0d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:41.331180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d2f2f-1594-47ee-95ef-51fe5c1ac19a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:41.345856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:41.373505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352109f-d9af-4fc1-ae7f-f1febcdacefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:41.374755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352109f-d9af-4fc1-ae7f-f1febcdacefd None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:41.375224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352109f-d9af-4fc1-ae7f-f1febcdacefd None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:41.375531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352109f-d9af-4fc1-ae7f-f1febcdacefd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:41.388222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352109f-d9af-4fc1-ae7f-f1febcdacefd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4352109f-d9af-4fc1-ae7f-f1febcdacefd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:41.393902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:41.394847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:41.396484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 510/1047] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:41.401082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352109f-d9af-4fc1-ae7f-f1febcdacefd None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:41.450778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d2f2f-1594-47ee-95ef-51fe5c1ac19a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:41.490971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc409842-d8d1-43d8-99db-48ea4eb4e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:41.491967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc409842-d8d1-43d8-99db-48ea4eb4e13a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:41.492263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc409842-d8d1-43d8-99db-48ea4eb4e13a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:41.492546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc409842-d8d1-43d8-99db-48ea4eb4e13a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:41.493376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc409842-d8d1-43d8-99db-48ea4eb4e13a None admin] RBAC: Adding query filter params (role.id=7945ffd9805a4831932cab7339f7993c, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=5706aa2d6bf54f05badf9588f51f4677) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:41.493693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc409842-d8d1-43d8-99db-48ea4eb4e13a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:41.495817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc409842-d8d1-43d8-99db-48ea4eb4e13a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:41.507064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352109f-d9af-4fc1-ae7f-f1febcdacefd None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:41.509980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 538/1048] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:41 2022] GET /identity/v3/role_assignments?user.id=5706aa2d6bf54f05badf9588f51f4677&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=7945ffd9805a4831932cab7339f7993c => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:41.548525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67013a8-5f95-4b21-84be-f969aadce054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:41.549212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67013a8-5f95-4b21-84be-f969aadce054 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:41.549546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67013a8-5f95-4b21-84be-f969aadce054 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:41.549826 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67013a8-5f95-4b21-84be-f969aadce054 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:41.550564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67013a8-5f95-4b21-84be-f969aadce054 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:41.550867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67013a8-5f95-4b21-84be-f969aadce054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:41.552287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67013a8-5f95-4b21-84be-f969aadce054 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:41.562372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 511/1049] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:10:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:41.568264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc409842-d8d1-43d8-99db-48ea4eb4e13a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:41.595242 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:41.682858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc409842-d8d1-43d8-99db-48ea4eb4e13a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:41.720653 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94d9c0-32e6-43cf-9b88-03e78f16d06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:41.721314 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94d9c0-32e6-43cf-9b88-03e78f16d06c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:41.721607 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94d9c0-32e6-43cf-9b88-03e78f16d06c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:41.721885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94d9c0-32e6-43cf-9b88-03e78f16d06c None admin] PATH_INFO: `/v3/users/2fe7b5d5aad34397a779645e9ad7dc76` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:41.736781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94d9c0-32e6-43cf-9b88-03e78f16d06c None admin] RBAC: Authorizing `identity:get_user(user_id=2fe7b5d5aad34397a779645e9ad7dc76)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:41.738114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94d9c0-32e6-43cf-9b88-03e78f16d06c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:41.744546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 539/1050] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:10:41 2022] GET /identity/v3/users/2fe7b5d5aad34397a779645e9ad7dc76 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:41.745786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67013a8-5f95-4b21-84be-f969aadce054 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:41.785827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:41.856001 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67013a8-5f95-4b21-84be-f969aadce054 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:41.891163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ae878-4be2-42c9-ad4d-b43ba48724e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:41.891791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ae878-4be2-42c9-ad4d-b43ba48724e6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:41.892117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ae878-4be2-42c9-ad4d-b43ba48724e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:41.892403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ae878-4be2-42c9-ad4d-b43ba48724e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:41.961905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ae878-4be2-42c9-ad4d-b43ba48724e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:41.963289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ae878-4be2-42c9-ad4d-b43ba48724e6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:41.998271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:42.073499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 512/1051] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:10:42.077981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94d9c0-32e6-43cf-9b88-03e78f16d06c None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:42.120274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60514b82-22bc-4f66-84d3-af364951f6f1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:42.120614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60514b82-22bc-4f66-84d3-af364951f6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:42.120845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60514b82-22bc-4f66-84d3-af364951f6f1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:42.122381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 513/1052] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:42.127558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b92dd-e026-4b51-b235-6ad9e4b6dffa None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:42.127851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b92dd-e026-4b51-b235-6ad9e4b6dffa None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:42.128153 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b92dd-e026-4b51-b235-6ad9e4b6dffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:42.148319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a94b92dd-e026-4b51-b235-6ad9e4b6dffa None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:42.149704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a94b92dd-e026-4b51-b235-6ad9e4b6dffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:42.164327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:42.187554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94d9c0-32e6-43cf-9b88-03e78f16d06c None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:42.226584 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 514/1053] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:42.229733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cd346-fa32-4479-9bf5-a59deabe7ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:42.230629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cd346-fa32-4479-9bf5-a59deabe7ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:42.231104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cd346-fa32-4479-9bf5-a59deabe7ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:42.231567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cd346-fa32-4479-9bf5-a59deabe7ce0 None admin] PATH_INFO: `/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:42.232967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4cd346-fa32-4479-9bf5-a59deabe7ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=a1dea3ca8e8f4df987f62c98bb0ffe26)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:42.234841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4cd346-fa32-4479-9bf5-a59deabe7ce0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:42.236771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 540/1054] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:10:42 2022] GET /identity/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:42.243235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b92dd-e026-4b51-b235-6ad9e4b6dffa None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:42.254729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cd346-fa32-4479-9bf5-a59deabe7ce0 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:42.337359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:42.381920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b92dd-e026-4b51-b235-6ad9e4b6dffa None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:42.426759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d103d2b-3425-417f-8c91-347405099ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:42.427384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d103d2b-3425-417f-8c91-347405099ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:42.427663 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d103d2b-3425-417f-8c91-347405099ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:42.428141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d103d2b-3425-417f-8c91-347405099ee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:42.428776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d103d2b-3425-417f-8c91-347405099ee2 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=a1dea3ca8e8f4df987f62c98bb0ffe26, user.id=2fe7b5d5aad34397a779645e9ad7dc76) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:42.429034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d103d2b-3425-417f-8c91-347405099ee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:42.430515 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d103d2b-3425-417f-8c91-347405099ee2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:42.431195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cd346-fa32-4479-9bf5-a59deabe7ce0 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:42.445342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 515/1055] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:42 2022] GET /identity/v3/role_assignments?user.id=2fe7b5d5aad34397a779645e9ad7dc76&scope.project.id=a1dea3ca8e8f4df987f62c98bb0ffe26&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:42.483473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fcb37-d19a-45a7-81ce-64ae58d4018f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:42.484083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fcb37-d19a-45a7-81ce-64ae58d4018f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:42.484384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fcb37-d19a-45a7-81ce-64ae58d4018f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:42.484671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fcb37-d19a-45a7-81ce-64ae58d4018f None admin] PATH_INFO: `/v3/services/image` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:42.485365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fcb37-d19a-45a7-81ce-64ae58d4018f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:42.486797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fcb37-d19a-45a7-81ce-64ae58d4018f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-539fcb37-d19a-45a7-81ce-64ae58d4018f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:42.498021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:42.499679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:42.506278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 541/1056] 10.222.0.69 () {56 vars in 1105 bytes} [Thu Jul 28 06:10:42 2022] GET /identity/v3/services/image => generated 86 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:42.507714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d103d2b-3425-417f-8c91-347405099ee2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:42.638209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d103d2b-3425-417f-8c91-347405099ee2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:42.668932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:42.675876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a59c67-4110-48c4-826e-378669157bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:42.676510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a59c67-4110-48c4-826e-378669157bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:42.676798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a59c67-4110-48c4-826e-378669157bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:42.677079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a59c67-4110-48c4-826e-378669157bf3 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:42.677851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a59c67-4110-48c4-826e-378669157bf3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:42.678124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a59c67-4110-48c4-826e-378669157bf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:42.679634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a59c67-4110-48c4-826e-378669157bf3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:42.685069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 516/1057] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:42.691372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fcb37-d19a-45a7-81ce-64ae58d4018f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:42.706497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:42.807702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fcb37-d19a-45a7-81ce-64ae58d4018f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:42.844775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c266f4-47c0-4121-b0e5-5a4339f7fa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:42.845439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c266f4-47c0-4121-b0e5-5a4339f7fa43 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:42.845761 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c266f4-47c0-4121-b0e5-5a4339f7fa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:42.846045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c266f4-47c0-4121-b0e5-5a4339f7fa43 None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:42.846740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c266f4-47c0-4121-b0e5-5a4339f7fa43 None admin] RBAC: Adding query filter params (type=image) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:42.847050 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c266f4-47c0-4121-b0e5-5a4339f7fa43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:42.848448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c266f4-47c0-4121-b0e5-5a4339f7fa43 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:42.859284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 542/1058] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:42.860506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a59c67-4110-48c4-826e-378669157bf3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:42.965048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a59c67-4110-48c4-826e-378669157bf3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:43.000095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffe221-a3ff-4ec8-8892-aafe19dc4f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:43.000720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffe221-a3ff-4ec8-8892-aafe19dc4f8e None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:43.001001 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffe221-a3ff-4ec8-8892-aafe19dc4f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:43.001276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffe221-a3ff-4ec8-8892-aafe19dc4f8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:43.002014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffe221-a3ff-4ec8-8892-aafe19dc4f8e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:43.003381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffe221-a3ff-4ec8-8892-aafe19dc4f8e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:43.104657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 517/1059] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:10:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:43.664214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d24d5e-2fb6-4b81-83ce-df8f20240524 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:43.664521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d24d5e-2fb6-4b81-83ce-df8f20240524 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:43.664706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d24d5e-2fb6-4b81-83ce-df8f20240524 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:43.666345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 543/1060] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:43.672021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e92fe-729b-43ae-8cf3-a92a7acd3658 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:43.672510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e92fe-729b-43ae-8cf3-a92a7acd3658 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:43.672587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e92fe-729b-43ae-8cf3-a92a7acd3658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:43.694111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fb8e92fe-729b-43ae-8cf3-a92a7acd3658 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:43.695322 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fb8e92fe-729b-43ae-8cf3-a92a7acd3658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:43.747247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:43.777510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 518/1061] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:43.799604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d24d5e-2fb6-4b81-83ce-df8f20240524 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:43.882582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:43.909820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d24d5e-2fb6-4b81-83ce-df8f20240524 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:43.947460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379eb03-eca4-4cfb-9921-bed4277dbb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:43.948041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379eb03-eca4-4cfb-9921-bed4277dbb2b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:43.948332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379eb03-eca4-4cfb-9921-bed4277dbb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:43.948617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379eb03-eca4-4cfb-9921-bed4277dbb2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:43.960108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379eb03-eca4-4cfb-9921-bed4277dbb2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-9379eb03-eca4-4cfb-9921-bed4277dbb2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:43.964989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:43.966073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:43.972334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 544/1062] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 28 06:10:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:43.974735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e92fe-729b-43ae-8cf3-a92a7acd3658 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:44.076823 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e92fe-729b-43ae-8cf3-a92a7acd3658 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:44.112994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b2d1f-2e24-4110-86f5-1a81f12928d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:44.113649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b2d1f-2e24-4110-86f5-1a81f12928d4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:44.113939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b2d1f-2e24-4110-86f5-1a81f12928d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:44.114218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b2d1f-2e24-4110-86f5-1a81f12928d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:44.114941 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b2d1f-2e24-4110-86f5-1a81f12928d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:44.115252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b2d1f-2e24-4110-86f5-1a81f12928d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:44.116637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b2d1f-2e24-4110-86f5-1a81f12928d4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:44.129881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 519/1063] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 28 06:10:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:44.131504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379eb03-eca4-4cfb-9921-bed4277dbb2b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:44.243040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379eb03-eca4-4cfb-9921-bed4277dbb2b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:44.280388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78013a-944d-47be-a4a6-9289899c3fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:44.281061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78013a-944d-47be-a4a6-9289899c3fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:44.281377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78013a-944d-47be-a4a6-9289899c3fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:44.281665 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78013a-944d-47be-a4a6-9289899c3fa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:44.283880 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:44.293986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78013a-944d-47be-a4a6-9289899c3fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:44.295501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78013a-944d-47be-a4a6-9289899c3fa2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-2c78013a-944d-47be-a4a6-9289899c3fa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:44.310130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:44.311252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:44.312784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:44.312784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:44.312784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:44.312784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:44.316820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 545/1064] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:44.317900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b2d1f-2e24-4110-86f5-1a81f12928d4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:44.367469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:44.435043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b2d1f-2e24-4110-86f5-1a81f12928d4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:44.480162 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1d7eb-5dc0-44f0-b42c-c859bcba74c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:44.480918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1d7eb-5dc0-44f0-b42c-c859bcba74c8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:44.481202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1d7eb-5dc0-44f0-b42c-c859bcba74c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:44.481508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1d7eb-5dc0-44f0-b42c-c859bcba74c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:44.495995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1d7eb-5dc0-44f0-b42c-c859bcba74c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:44.495995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1d7eb-5dc0-44f0-b42c-c859bcba74c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:44.495995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1d7eb-5dc0-44f0-b42c-c859bcba74c8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:44.502078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 520/1065] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:44.503324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78013a-944d-47be-a4a6-9289899c3fa2 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:44.610950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78013a-944d-47be-a4a6-9289899c3fa2 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:44.644979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78942786-2d06-4918-8a7a-db6e59fbaad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:44.645666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78942786-2d06-4918-8a7a-db6e59fbaad9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:44.645948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78942786-2d06-4918-8a7a-db6e59fbaad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:44.646223 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78942786-2d06-4918-8a7a-db6e59fbaad9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:44.648571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b29642-b9c8-43cf-befd-49d6153b20f6 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:44.648855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b29642-b9c8-43cf-befd-49d6153b20f6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:44.649134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b29642-b9c8-43cf-befd-49d6153b20f6 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:44.650773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 521/1066] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:10:44.656416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78942786-2d06-4918-8a7a-db6e59fbaad9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:44.656553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d63175-48c4-409a-b5c3-d2b1e8251994 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:44.656846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d63175-48c4-409a-b5c3-d2b1e8251994 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:44.657169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d63175-48c4-409a-b5c3-d2b1e8251994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:44.657905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78942786-2d06-4918-8a7a-db6e59fbaad9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-78942786-2d06-4918-8a7a-db6e59fbaad9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:44.670327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:44.671352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:44.672307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 546/1067] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 28 06:10:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:44.677432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78942786-2d06-4918-8a7a-db6e59fbaad9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:44.677693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-15d63175-48c4-409a-b5c3-d2b1e8251994 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:44.678898 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-15d63175-48c4-409a-b5c3-d2b1e8251994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:44.728614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:44.755033 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 522/1068] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:44.777538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d63175-48c4-409a-b5c3-d2b1e8251994 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:44.793336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78942786-2d06-4918-8a7a-db6e59fbaad9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:44.810267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:44.837787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fcde4-4e5f-4ed0-a790-966f4deef551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:44.838460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fcde4-4e5f-4ed0-a790-966f4deef551 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:44.838700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fcde4-4e5f-4ed0-a790-966f4deef551 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:44.838997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fcde4-4e5f-4ed0-a790-966f4deef551 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:44.839781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fcde4-4e5f-4ed0-a790-966f4deef551 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:44.840092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fcde4-4e5f-4ed0-a790-966f4deef551 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:44.841540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fcde4-4e5f-4ed0-a790-966f4deef551 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:44.861649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 547/1069] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 28 06:10:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:44.867558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fcde4-4e5f-4ed0-a790-966f4deef551 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:44.897113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d63175-48c4-409a-b5c3-d2b1e8251994 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:44.943842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72ed3d-75cb-4e29-8a52-35577e928d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:44.944621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72ed3d-75cb-4e29-8a52-35577e928d49 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:44.944882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72ed3d-75cb-4e29-8a52-35577e928d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:44.945170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72ed3d-75cb-4e29-8a52-35577e928d49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:44.958119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72ed3d-75cb-4e29-8a52-35577e928d49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-7f72ed3d-75cb-4e29-8a52-35577e928d49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:44.963428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:44.964454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:44.965280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 523/1070] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:44.971463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72ed3d-75cb-4e29-8a52-35577e928d49 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:44.989670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fcde4-4e5f-4ed0-a790-966f4deef551 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:44.992397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:45.028531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f131378-1fb9-4ec3-847a-ead4e1b50fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:45.029110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f131378-1fb9-4ec3-847a-ead4e1b50fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.029430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f131378-1fb9-4ec3-847a-ead4e1b50fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.029697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f131378-1fb9-4ec3-847a-ead4e1b50fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.040318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f131378-1fb9-4ec3-847a-ead4e1b50fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:45.041819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f131378-1fb9-4ec3-847a-ead4e1b50fe5 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-9f131378-1fb9-4ec3-847a-ead4e1b50fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:45.056940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:45.058061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:45.058976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:45.058976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:45.058976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:45.058976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:45.063541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 548/1071] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:45.064217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f131378-1fb9-4ec3-847a-ead4e1b50fe5 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:45.068931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:45.081567 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72ed3d-75cb-4e29-8a52-35577e928d49 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:45.121965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c0bf8-44a0-4fea-86d9-b3ca2b6d0b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:45.122593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c0bf8-44a0-4fea-86d9-b3ca2b6d0b89 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.122882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c0bf8-44a0-4fea-86d9-b3ca2b6d0b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.123162 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c0bf8-44a0-4fea-86d9-b3ca2b6d0b89 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.123917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c0bf8-44a0-4fea-86d9-b3ca2b6d0b89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:45.124235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c0bf8-44a0-4fea-86d9-b3ca2b6d0b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:45.125724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c0bf8-44a0-4fea-86d9-b3ca2b6d0b89 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:45.134821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 524/1072] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:10:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:45.140240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c0bf8-44a0-4fea-86d9-b3ca2b6d0b89 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:45.186377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f131378-1fb9-4ec3-847a-ead4e1b50fe5 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:45.222305 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77052c03-88d2-4bc0-b200-e011a8b8fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:45.222926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77052c03-88d2-4bc0-b200-e011a8b8fc44 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.223213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77052c03-88d2-4bc0-b200-e011a8b8fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.223491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77052c03-88d2-4bc0-b200-e011a8b8fc44 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.224199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77052c03-88d2-4bc0-b200-e011a8b8fc44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:45.224504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77052c03-88d2-4bc0-b200-e011a8b8fc44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:45.225919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77052c03-88d2-4bc0-b200-e011a8b8fc44 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:45.238149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 549/1073] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:45.239375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735d191-2d71-4f55-9d19-33f865e5523c None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.239662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735d191-2d71-4f55-9d19-33f865e5523c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.239938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735d191-2d71-4f55-9d19-33f865e5523c None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.241456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 550/1074] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:45.243507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735d191-2d71-4f55-9d19-33f865e5523c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:45.251185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c0bf8-44a0-4fea-86d9-b3ca2b6d0b89 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:45.261568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:45.287167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e7981-402b-4741-bfc3-90bb911e87a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:45.287652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e7981-402b-4741-bfc3-90bb911e87a1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.287946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e7981-402b-4741-bfc3-90bb911e87a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.288222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e7981-402b-4741-bfc3-90bb911e87a1 None admin] PATH_INFO: `/v3/users/2fe7b5d5aad34397a779645e9ad7dc76` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.289481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e7981-402b-4741-bfc3-90bb911e87a1 None admin] RBAC: Authorizing `identity:get_user(user_id=2fe7b5d5aad34397a779645e9ad7dc76)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:45.290935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e7981-402b-4741-bfc3-90bb911e87a1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:45.292722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 525/1075] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:10:45 2022] GET /identity/v3/users/2fe7b5d5aad34397a779645e9ad7dc76 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:45.293708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629df0-3a72-4d6b-bbaa-c2289b684a65 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.294000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629df0-3a72-4d6b-bbaa-c2289b684a65 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.294278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629df0-3a72-4d6b-bbaa-c2289b684a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.312189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-46629df0-3a72-4d6b-bbaa-c2289b684a65 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:45.313662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-46629df0-3a72-4d6b-bbaa-c2289b684a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:45.326723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:45.354233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735d191-2d71-4f55-9d19-33f865e5523c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:45.384601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 526/1076] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:45.385248 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46629df0-3a72-4d6b-bbaa-c2289b684a65 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:45.391086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84a146-f4e7-4159-8846-bf15829106ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:45.391733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84a146-f4e7-4159-8846-bf15829106ba None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.392022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84a146-f4e7-4159-8846-bf15829106ba None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.392302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84a146-f4e7-4159-8846-bf15829106ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.399085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84a146-f4e7-4159-8846-bf15829106ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:45.401212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84a146-f4e7-4159-8846-bf15829106ba None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-1e84a146-f4e7-4159-8846-bf15829106ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:45.413939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:45.414997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:45.420821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:45.420821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:45.420821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:45.420821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:45.420821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:45.420821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:45.420821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:45.420821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:45.420821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 551/1077] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:45.420821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84a146-f4e7-4159-8846-bf15829106ba None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:45.494119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:45.499315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46629df0-3a72-4d6b-bbaa-c2289b684a65 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:45.518512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84a146-f4e7-4159-8846-bf15829106ba None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:45.535450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521449aa-29b5-41be-a867-ab5662a5b51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:45.536090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521449aa-29b5-41be-a867-ab5662a5b51b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.536370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521449aa-29b5-41be-a867-ab5662a5b51b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.536645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521449aa-29b5-41be-a867-ab5662a5b51b None admin] PATH_INFO: `/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.537887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521449aa-29b5-41be-a867-ab5662a5b51b None admin] RBAC: Authorizing `identity:get_project(project_id=a1dea3ca8e8f4df987f62c98bb0ffe26)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:45.539385 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521449aa-29b5-41be-a867-ab5662a5b51b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:45.541200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 527/1078] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:10:45 2022] GET /identity/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:45.541825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521449aa-29b5-41be-a867-ab5662a5b51b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:45.558555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511912ed-2d8a-4ac9-949d-601565afa71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:45.559180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511912ed-2d8a-4ac9-949d-601565afa71f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.559474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511912ed-2d8a-4ac9-949d-601565afa71f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.559669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:45.559780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511912ed-2d8a-4ac9-949d-601565afa71f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.626742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511912ed-2d8a-4ac9-949d-601565afa71f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:45.628205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511912ed-2d8a-4ac9-949d-601565afa71f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:45.641371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521449aa-29b5-41be-a867-ab5662a5b51b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:45.675073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a7cf5-1080-4aff-a999-2ebae3417159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:45.675693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a7cf5-1080-4aff-a999-2ebae3417159 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.675984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a7cf5-1080-4aff-a999-2ebae3417159 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.676263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a7cf5-1080-4aff-a999-2ebae3417159 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.677018 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a7cf5-1080-4aff-a999-2ebae3417159 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:45.677362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a7cf5-1080-4aff-a999-2ebae3417159 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:45.678807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a7cf5-1080-4aff-a999-2ebae3417159 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:45.690849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 528/1079] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:45.691720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a7cf5-1080-4aff-a999-2ebae3417159 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:45.704558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:45.728543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 552/1080] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:10:45.729372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511912ed-2d8a-4ac9-949d-601565afa71f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:45.776803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:45.801595 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a7cf5-1080-4aff-a999-2ebae3417159 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:45.835652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78c72d-ce54-41c7-8f57-59c63daaa6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:45.836334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78c72d-ce54-41c7-8f57-59c63daaa6ad None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.836618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78c72d-ce54-41c7-8f57-59c63daaa6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.836895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78c72d-ce54-41c7-8f57-59c63daaa6ad None admin] PATH_INFO: `/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26/users/2fe7b5d5aad34397a779645e9ad7dc76/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.838882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78c72d-ce54-41c7-8f57-59c63daaa6ad None admin] RBAC: Authorizing `identity:create_grant(project_id=a1dea3ca8e8f4df987f62c98bb0ffe26, user_id=2fe7b5d5aad34397a779645e9ad7dc76, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:45.838987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511912ed-2d8a-4ac9-949d-601565afa71f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:45.840341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78c72d-ce54-41c7-8f57-59c63daaa6ad None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:45.874861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac004761-2b7f-4a75-8bb9-29b18fa64a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:45.875540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac004761-2b7f-4a75-8bb9-29b18fa64a41 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:45.875834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac004761-2b7f-4a75-8bb9-29b18fa64a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:45.876119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac004761-2b7f-4a75-8bb9-29b18fa64a41 None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/5706aa2d6bf54f05badf9588f51f4677/roles/7945ffd9805a4831932cab7339f7993c` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:45.878329 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac004761-2b7f-4a75-8bb9-29b18fa64a41 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=5706aa2d6bf54f05badf9588f51f4677, role_id=7945ffd9805a4831932cab7339f7993c)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:45.879772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac004761-2b7f-4a75-8bb9-29b18fa64a41 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:45.901574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 529/1081] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:10:45 2022] PUT /identity/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26/users/2fe7b5d5aad34397a779645e9ad7dc76/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:10:45.902408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78c72d-ce54-41c7-8f57-59c63daaa6ad None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:46.010796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 553/1082] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:10:45 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/5706aa2d6bf54f05badf9588f51f4677/roles/7945ffd9805a4831932cab7339f7993c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:10:46.019775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78c72d-ce54-41c7-8f57-59c63daaa6ad None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:46.066150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415df683-f611-4f78-b8eb-a628a1949653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:46.066752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415df683-f611-4f78-b8eb-a628a1949653 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:46.067044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415df683-f611-4f78-b8eb-a628a1949653 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:46.067409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415df683-f611-4f78-b8eb-a628a1949653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:46.071404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:46.081365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415df683-f611-4f78-b8eb-a628a1949653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:46.082890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415df683-f611-4f78-b8eb-a628a1949653 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-415df683-f611-4f78-b8eb-a628a1949653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:46.100059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:46.101085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:46.102104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:46.102104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:46.102104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:46.102104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:46.102104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 530/1083] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:46.107956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac004761-2b7f-4a75-8bb9-29b18fa64a41 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:46.154395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:46.231606 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac004761-2b7f-4a75-8bb9-29b18fa64a41 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:46.268718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6d4d0-39fa-4d78-84fc-f65e203a6b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:46.269627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6d4d0-39fa-4d78-84fc-f65e203a6b21 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:46.270063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6d4d0-39fa-4d78-84fc-f65e203a6b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:46.270524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6d4d0-39fa-4d78-84fc-f65e203a6b21 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:46.271378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6d4d0-39fa-4d78-84fc-f65e203a6b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:46.271833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6d4d0-39fa-4d78-84fc-f65e203a6b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:46.273386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6d4d0-39fa-4d78-84fc-f65e203a6b21 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:46.286137 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 554/1084] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:47.977773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8ecf-32aa-444c-ad49-4146e439907f None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:47.978310 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8ecf-32aa-444c-ad49-4146e439907f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:47.978400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8ecf-32aa-444c-ad49-4146e439907f None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:47.979853 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 531/1085] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:47.985388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deb712-56f1-4300-b477-854181fa3689 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:47.985798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deb712-56f1-4300-b477-854181fa3689 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:47.986082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deb712-56f1-4300-b477-854181fa3689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.004138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b3deb712-56f1-4300-b477-854181fa3689 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:48.005356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b3deb712-56f1-4300-b477-854181fa3689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:48.059520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f90f3-d44a-430e-9546-6e5a37414cbb None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:48.059812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f90f3-d44a-430e-9546-6e5a37414cbb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:48.060095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f90f3-d44a-430e-9546-6e5a37414cbb None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.065336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 532/1086] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:48.066782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcf7c4-612d-4b3f-b805-f76c38efa58f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:48.067257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcf7c4-612d-4b3f-b805-f76c38efa58f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:48.067543 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcf7c4-612d-4b3f-b805-f76c38efa58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.072040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 555/1087] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:48.086419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ecfcf7c4-612d-4b3f-b805-f76c38efa58f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:48.087608 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ecfcf7c4-612d-4b3f-b805-f76c38efa58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:48.094993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3deb712-56f1-4300-b477-854181fa3689 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:48.122099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:48.158996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 533/1088] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:10:48.182475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcf7c4-612d-4b3f-b805-f76c38efa58f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:48.198010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3deb712-56f1-4300-b477-854181fa3689 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:48.232477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774f391-a0f6-4c01-935b-14a3891c2792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:48.233634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774f391-a0f6-4c01-935b-14a3891c2792 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:48.233936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774f391-a0f6-4c01-935b-14a3891c2792 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:48.234208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774f391-a0f6-4c01-935b-14a3891c2792 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.244076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774f391-a0f6-4c01-935b-14a3891c2792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-a774f391-a0f6-4c01-935b-14a3891c2792 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:48.249578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:48.250529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:48.251686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 556/1089] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:10:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:48.256841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774f391-a0f6-4c01-935b-14a3891c2792 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:48.282530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcf7c4-612d-4b3f-b805-f76c38efa58f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:48.301780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:48.319282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f4829-af8d-4992-8b68-714484731b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:48.319926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f4829-af8d-4992-8b68-714484731b4f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:48.320207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f4829-af8d-4992-8b68-714484731b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:48.320481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f4829-af8d-4992-8b68-714484731b4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.330894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f4829-af8d-4992-8b68-714484731b4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a25f4829-af8d-4992-8b68-714484731b4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:10:48.336006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:48.336949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:48.337850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 534/1090] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 28 06:10:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:48.343581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f4829-af8d-4992-8b68-714484731b4f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:48.348016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:48.364268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774f391-a0f6-4c01-935b-14a3891c2792 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:48.401010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a326f03-c6bc-4ca7-a2f4-a4da4620f0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:48.401670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a326f03-c6bc-4ca7-a2f4-a4da4620f0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:48.401959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a326f03-c6bc-4ca7-a2f4-a4da4620f0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:48.402235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a326f03-c6bc-4ca7-a2f4-a4da4620f0c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.402957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a326f03-c6bc-4ca7-a2f4-a4da4620f0c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:48.403261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a326f03-c6bc-4ca7-a2f4-a4da4620f0c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:48.404648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a326f03-c6bc-4ca7-a2f4-a4da4620f0c2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:48.413765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 557/1091] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:10:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:48.419897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a326f03-c6bc-4ca7-a2f4-a4da4620f0c2 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:48.456920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f4829-af8d-4992-8b68-714484731b4f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:48.491652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436ad26-5a57-44b6-92eb-d0fa21ec852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:48.492185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436ad26-5a57-44b6-92eb-d0fa21ec852f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:48.492465 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436ad26-5a57-44b6-92eb-d0fa21ec852f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:48.492740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436ad26-5a57-44b6-92eb-d0fa21ec852f None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.493482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436ad26-5a57-44b6-92eb-d0fa21ec852f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:48.493785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436ad26-5a57-44b6-92eb-d0fa21ec852f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:48.495156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436ad26-5a57-44b6-92eb-d0fa21ec852f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:48.504331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 535/1092] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 28 06:10:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:48.510262 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436ad26-5a57-44b6-92eb-d0fa21ec852f None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:48.523806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a326f03-c6bc-4ca7-a2f4-a4da4620f0c2 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:48.553905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:48.558705 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a4bf9-6b97-438c-bcd6-8a898087234c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:48.559355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a4bf9-6b97-438c-bcd6-8a898087234c None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:48.559664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a4bf9-6b97-438c-bcd6-8a898087234c None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:48.559917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a4bf9-6b97-438c-bcd6-8a898087234c None admin] PATH_INFO: `/v3/users/2fe7b5d5aad34397a779645e9ad7dc76` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.561856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a4bf9-6b97-438c-bcd6-8a898087234c None admin] RBAC: Authorizing `identity:get_user(user_id=2fe7b5d5aad34397a779645e9ad7dc76)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:48.563493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a4bf9-6b97-438c-bcd6-8a898087234c None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:48.565118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 558/1093] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:10:48 2022] GET /identity/v3/users/2fe7b5d5aad34397a779645e9ad7dc76 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:48.570971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a4bf9-6b97-438c-bcd6-8a898087234c None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:48.592216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:48.613676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436ad26-5a57-44b6-92eb-d0fa21ec852f None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:48.650047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5acd48f-f8b3-4ffd-9983-db74c5df9260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:48.650699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5acd48f-f8b3-4ffd-9983-db74c5df9260 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:48.650977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5acd48f-f8b3-4ffd-9983-db74c5df9260 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:48.651251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5acd48f-f8b3-4ffd-9983-db74c5df9260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.663058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5acd48f-f8b3-4ffd-9983-db74c5df9260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:48.664522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5acd48f-f8b3-4ffd-9983-db74c5df9260 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:48.677371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a4bf9-6b97-438c-bcd6-8a898087234c None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-a5acd48f-f8b3-4ffd-9983-db74c5df9260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:48.678846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:48.679825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:48.681078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:48.681078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:48.681078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:48.681078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:48.681078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 536/1094] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:48.685740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5acd48f-f8b3-4ffd-9983-db74c5df9260 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:48.714718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59353a18-1831-4d3d-9ef4-f5be5488ef51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:48.715347 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59353a18-1831-4d3d-9ef4-f5be5488ef51 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:48.715633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59353a18-1831-4d3d-9ef4-f5be5488ef51 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:48.715913 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59353a18-1831-4d3d-9ef4-f5be5488ef51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.778394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:48.784236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59353a18-1831-4d3d-9ef4-f5be5488ef51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:48.785736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59353a18-1831-4d3d-9ef4-f5be5488ef51 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:48.787932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5acd48f-f8b3-4ffd-9983-db74c5df9260 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:48.802479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:48.822360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fd61a-34ba-4545-a3fd-f26b31928fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:48.822994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fd61a-34ba-4545-a3fd-f26b31928fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:48.823281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fd61a-34ba-4545-a3fd-f26b31928fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:48.823744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fd61a-34ba-4545-a3fd-f26b31928fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:48.824330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fd61a-34ba-4545-a3fd-f26b31928fd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:48.824641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fd61a-34ba-4545-a3fd-f26b31928fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:48.826068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fd61a-34ba-4545-a3fd-f26b31928fd7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:48.841551 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 537/1095] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:48.853591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fd61a-34ba-4545-a3fd-f26b31928fd7 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:48.899600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 559/1096] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 06:10:48.906345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59353a18-1831-4d3d-9ef4-f5be5488ef51 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:48.971931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fd61a-34ba-4545-a3fd-f26b31928fd7 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:49.007161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df842cce-0690-4b6a-830f-7f9dbcf7486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:49.007872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df842cce-0690-4b6a-830f-7f9dbcf7486d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:49.008271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df842cce-0690-4b6a-830f-7f9dbcf7486d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:49.008628 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df842cce-0690-4b6a-830f-7f9dbcf7486d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:49.011013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59353a18-1831-4d3d-9ef4-f5be5488ef51 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:49.016792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df842cce-0690-4b6a-830f-7f9dbcf7486d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:49.018388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df842cce-0690-4b6a-830f-7f9dbcf7486d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:49.028063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-df842cce-0690-4b6a-830f-7f9dbcf7486d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:10:49.031381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:49.032457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:10:49.033555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 538/1097] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 28 06:10:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:49.038836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df842cce-0690-4b6a-830f-7f9dbcf7486d None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:49.047279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40044946-8f22-41c3-92f1-57d9a22c1eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:49.047934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40044946-8f22-41c3-92f1-57d9a22c1eef None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:49.048227 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40044946-8f22-41c3-92f1-57d9a22c1eef None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:49.048522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40044946-8f22-41c3-92f1-57d9a22c1eef None admin] PATH_INFO: `/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:49.050026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40044946-8f22-41c3-92f1-57d9a22c1eef None admin] RBAC: Authorizing `identity:get_project(project_id=a1dea3ca8e8f4df987f62c98bb0ffe26)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:49.051532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40044946-8f22-41c3-92f1-57d9a22c1eef None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:49.053297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 560/1098] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:10:48 2022] GET /identity/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:49.054376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:49.059283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40044946-8f22-41c3-92f1-57d9a22c1eef None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:49.143348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df842cce-0690-4b6a-830f-7f9dbcf7486d None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:49.161321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40044946-8f22-41c3-92f1-57d9a22c1eef None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:49.180433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cb488-00d3-4e17-ab8f-61e950e4a89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:49.181092 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cb488-00d3-4e17-ab8f-61e950e4a89b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:49.181408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cb488-00d3-4e17-ab8f-61e950e4a89b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:49.182020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cb488-00d3-4e17-ab8f-61e950e4a89b None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:49.182836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cb488-00d3-4e17-ab8f-61e950e4a89b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:49.183151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cb488-00d3-4e17-ab8f-61e950e4a89b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:49.184649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cb488-00d3-4e17-ab8f-61e950e4a89b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:49.200293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b6372-074a-4ca4-9869-75f426f85bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:49.200990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b6372-074a-4ca4-9869-75f426f85bff None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:49.201326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b6372-074a-4ca4-9869-75f426f85bff None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:49.201647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b6372-074a-4ca4-9869-75f426f85bff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:49.203059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b6372-074a-4ca4-9869-75f426f85bff None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=a1dea3ca8e8f4df987f62c98bb0ffe26, user.id=2fe7b5d5aad34397a779645e9ad7dc76) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:49.203378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b6372-074a-4ca4-9869-75f426f85bff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:49.204407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 539/1099] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 28 06:10:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:49.204848 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b6372-074a-4ca4-9869-75f426f85bff None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:49.210003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cb488-00d3-4e17-ab8f-61e950e4a89b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:49.212449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 561/1100] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:49 2022] GET /identity/v3/role_assignments?user.id=2fe7b5d5aad34397a779645e9ad7dc76&scope.project.id=a1dea3ca8e8f4df987f62c98bb0ffe26&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:49.293049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:49.327075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cb488-00d3-4e17-ab8f-61e950e4a89b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:49.366082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428b65-cebb-42a6-a688-d1573c220e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:49.366259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428b65-cebb-42a6-a688-d1573c220e29 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:49.366544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428b65-cebb-42a6-a688-d1573c220e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:49.366816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428b65-cebb-42a6-a688-d1573c220e29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:49.437882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1428b65-cebb-42a6-a688-d1573c220e29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:49.439354 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1428b65-cebb-42a6-a688-d1573c220e29 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:49.506276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:49.540914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 540/1101] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 06:10:49.547217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b6372-074a-4ca4-9869-75f426f85bff None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:49.596877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:49.649255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b6372-074a-4ca4-9869-75f426f85bff None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:49.682477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6f934-ba97-480a-a235-ee17371d6411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:49.683113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6f934-ba97-480a-a235-ee17371d6411 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:49.683392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6f934-ba97-480a-a235-ee17371d6411 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:49.683667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6f934-ba97-480a-a235-ee17371d6411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:49.694937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6f934-ba97-480a-a235-ee17371d6411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:49.696383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6f934-ba97-480a-a235-ee17371d6411 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-dbd6f934-ba97-480a-a235-ee17371d6411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:10:49.711077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:49.712186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:49.713553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:10:49.713553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:10:49.713553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:10:49.713553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:49.713553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 562/1102] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:10:49.718785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428b65-cebb-42a6-a688-d1573c220e29 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:49.818000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428b65-cebb-42a6-a688-d1573c220e29 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:49.852424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8baad-2b19-42a7-a98a-3baf5e31ede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:49.853039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8baad-2b19-42a7-a98a-3baf5e31ede0 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:49.853349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8baad-2b19-42a7-a98a-3baf5e31ede0 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:49.853625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8baad-2b19-42a7-a98a-3baf5e31ede0 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:49.854316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8baad-2b19-42a7-a98a-3baf5e31ede0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:49.854615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8baad-2b19-42a7-a98a-3baf5e31ede0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:49.855982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8baad-2b19-42a7-a98a-3baf5e31ede0 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:49.867594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 541/1103] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:49.873229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6f934-ba97-480a-a235-ee17371d6411 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:49.970469 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:49.973474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6f934-ba97-480a-a235-ee17371d6411 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:50.006797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f525bb2-fc78-4cb5-b7ad-cb453dd86357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:50.007638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f525bb2-fc78-4cb5-b7ad-cb453dd86357 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:50.008246 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f525bb2-fc78-4cb5-b7ad-cb453dd86357 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:50.008532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f525bb2-fc78-4cb5-b7ad-cb453dd86357 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:50.016231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f525bb2-fc78-4cb5-b7ad-cb453dd86357 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:50.017722 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f525bb2-fc78-4cb5-b7ad-cb453dd86357 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-8f525bb2-fc78-4cb5-b7ad-cb453dd86357 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:10:50.028842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:10:50.030547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:10:50.031581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:10:50.031581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:10:50.031581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:10:50.031581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:10:50.031581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:10:50.031581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:10:50.031581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:10:50.031581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:10:50.031581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 563/1104] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:10:50.036567 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8baad-2b19-42a7-a98a-3baf5e31ede0 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:50.066975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:50.136246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8baad-2b19-42a7-a98a-3baf5e31ede0 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:50.169979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4189734-dc34-419f-87cd-66b513fd8483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:50.170727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4189734-dc34-419f-87cd-66b513fd8483 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:50.171016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4189734-dc34-419f-87cd-66b513fd8483 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:50.171297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4189734-dc34-419f-87cd-66b513fd8483 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:50.172051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4189734-dc34-419f-87cd-66b513fd8483 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:50.172363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4189734-dc34-419f-87cd-66b513fd8483 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:50.173853 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4189734-dc34-419f-87cd-66b513fd8483 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:50.185712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 542/1105] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:50.192264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f525bb2-fc78-4cb5-b7ad-cb453dd86357 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:50.292280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f525bb2-fc78-4cb5-b7ad-cb453dd86357 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:50.325961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b15f0a-245a-454a-afdd-06f4da03eccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:50.326584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b15f0a-245a-454a-afdd-06f4da03eccb None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:50.327765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b15f0a-245a-454a-afdd-06f4da03eccb None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:50.328050 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b15f0a-245a-454a-afdd-06f4da03eccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:50.328853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b15f0a-245a-454a-afdd-06f4da03eccb None admin] RBAC: Adding query filter params (role.id=7945ffd9805a4831932cab7339f7993c, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=5706aa2d6bf54f05badf9588f51f4677) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:50.329168 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b15f0a-245a-454a-afdd-06f4da03eccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:50.330649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b15f0a-245a-454a-afdd-06f4da03eccb None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:50.336605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 564/1106] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:50 2022] GET /identity/v3/role_assignments?user.id=5706aa2d6bf54f05badf9588f51f4677&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=7945ffd9805a4831932cab7339f7993c => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:51.273737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc70cb-9f74-42b5-821f-226ee2ab3f44 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:51.274351 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc70cb-9f74-42b5-821f-226ee2ab3f44 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:51.274351 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc70cb-9f74-42b5-821f-226ee2ab3f44 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:51.275652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 543/1107] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:51.281203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc39a7-fb05-48a6-b5bf-dfd4cdb287c7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:51.281694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc39a7-fb05-48a6-b5bf-dfd4cdb287c7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:51.281787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc39a7-fb05-48a6-b5bf-dfd4cdb287c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:51.300566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-98bc39a7-fb05-48a6-b5bf-dfd4cdb287c7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:51.301790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-98bc39a7-fb05-48a6-b5bf-dfd4cdb287c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:51.319780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:51.369320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 565/1108] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:51.394397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc70cb-9f74-42b5-821f-226ee2ab3f44 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:51.485162 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:51.495753 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc70cb-9f74-42b5-821f-226ee2ab3f44 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:51.532149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52975d1a-5ad3-437e-90ea-360f59ed1f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:51.532714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52975d1a-5ad3-437e-90ea-360f59ed1f5d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:51.532996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52975d1a-5ad3-437e-90ea-360f59ed1f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:51.533278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52975d1a-5ad3-437e-90ea-360f59ed1f5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:51.534453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52975d1a-5ad3-437e-90ea-360f59ed1f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:51.535887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52975d1a-5ad3-437e-90ea-360f59ed1f5d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:51.537571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 544/1109] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:10:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:10:51.543347 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc39a7-fb05-48a6-b5bf-dfd4cdb287c7 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:51.641611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc39a7-fb05-48a6-b5bf-dfd4cdb287c7 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:51.674709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5c172-9c31-4fb5-b4fd-5996dbadc6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:51.675328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5c172-9c31-4fb5-b4fd-5996dbadc6ec None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:51.675612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5c172-9c31-4fb5-b4fd-5996dbadc6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:51.675889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5c172-9c31-4fb5-b4fd-5996dbadc6ec None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:51.676578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5c172-9c31-4fb5-b4fd-5996dbadc6ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:51.678025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5c172-9c31-4fb5-b4fd-5996dbadc6ec None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:51.787519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 566/1110] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:10:51 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:10:53.836927 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351b918-8892-4e79-a618-b7fab9afe6a7 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:53.837528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351b918-8892-4e79-a618-b7fab9afe6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:53.837528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351b918-8892-4e79-a618-b7fab9afe6a7 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:53.838842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 545/1111] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:53.844064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd42e8-ee1d-4b8b-a331-1e19bb0f85d4 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:53.844498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd42e8-ee1d-4b8b-a331-1e19bb0f85d4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:53.844625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd42e8-ee1d-4b8b-a331-1e19bb0f85d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:53.863364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-44fd42e8-ee1d-4b8b-a331-1e19bb0f85d4 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:53.864582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-44fd42e8-ee1d-4b8b-a331-1e19bb0f85d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:53.867700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:53.933608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 567/1112] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:53.957052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351b918-8892-4e79-a618-b7fab9afe6a7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:54.059949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351b918-8892-4e79-a618-b7fab9afe6a7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:54.094712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afcd43-b6af-47dc-89fc-4f6ad06485ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:54.096123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afcd43-b6af-47dc-89fc-4f6ad06485ac None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:54.096123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afcd43-b6af-47dc-89fc-4f6ad06485ac None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:54.096123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afcd43-b6af-47dc-89fc-4f6ad06485ac None admin] PATH_INFO: `/v3/roles/c7b4bfc82f3b46cfaa10773dad7cd03a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:54.097552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17afcd43-b6af-47dc-89fc-4f6ad06485ac None admin] RBAC: Authorizing `identity:get_role(role_id=c7b4bfc82f3b46cfaa10773dad7cd03a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:54.098673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17afcd43-b6af-47dc-89fc-4f6ad06485ac None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:54.100029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 546/1113] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:10:53 2022] GET /identity/v3/roles/c7b4bfc82f3b46cfaa10773dad7cd03a => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:54.105986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd42e8-ee1d-4b8b-a331-1e19bb0f85d4 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:54.204405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd42e8-ee1d-4b8b-a331-1e19bb0f85d4 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:54.238573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11024a66-7def-4e7b-80bb-24ef49f74726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:54.239213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11024a66-7def-4e7b-80bb-24ef49f74726 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:54.239504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11024a66-7def-4e7b-80bb-24ef49f74726 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:54.239787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11024a66-7def-4e7b-80bb-24ef49f74726 None admin] PATH_INFO: `/v3/users/ecaf2631bf924cbd9bb4a2cd9d812c17` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:54.250020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:54.253375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11024a66-7def-4e7b-80bb-24ef49f74726 None admin] RBAC: Authorizing `identity:get_user(user_id=ecaf2631bf924cbd9bb4a2cd9d812c17)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:54.254839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11024a66-7def-4e7b-80bb-24ef49f74726 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:54.256515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 568/1114] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:10:54 2022] GET /identity/v3/users/ecaf2631bf924cbd9bb4a2cd9d812c17 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:54.262735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afcd43-b6af-47dc-89fc-4f6ad06485ac None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:54.291570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:54.365485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afcd43-b6af-47dc-89fc-4f6ad06485ac None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:54.405719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdc25d-2d4b-4669-a0bf-883288c511ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:54.406341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdc25d-2d4b-4669-a0bf-883288c511ce None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:54.406625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdc25d-2d4b-4669-a0bf-883288c511ce None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:54.406903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdc25d-2d4b-4669-a0bf-883288c511ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:54.473335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdc25d-2d4b-4669-a0bf-883288c511ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:54.474818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdc25d-2d4b-4669-a0bf-883288c511ce None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:54.502156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:54.574821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 547/1115] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:10:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:10:54.581381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11024a66-7def-4e7b-80bb-24ef49f74726 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:54.683120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11024a66-7def-4e7b-80bb-24ef49f74726 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:54.716692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643fa8a-fb63-4614-8201-0f7cbdfdc9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:54.717359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643fa8a-fb63-4614-8201-0f7cbdfdc9bd None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:54.717652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643fa8a-fb63-4614-8201-0f7cbdfdc9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:54.717936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643fa8a-fb63-4614-8201-0f7cbdfdc9bd None admin] PATH_INFO: `/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:54.719094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643fa8a-fb63-4614-8201-0f7cbdfdc9bd None admin] RBAC: Authorizing `identity:get_project(project_id=a1dea3ca8e8f4df987f62c98bb0ffe26)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:54.720528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643fa8a-fb63-4614-8201-0f7cbdfdc9bd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:54.722215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 569/1116] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:10:54 2022] GET /identity/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:54.728344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdc25d-2d4b-4669-a0bf-883288c511ce None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:54.828407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdc25d-2d4b-4669-a0bf-883288c511ce None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:54.854919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:54.864457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a89f92-43c2-4f68-8ed4-8122338dca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:54.865081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a89f92-43c2-4f68-8ed4-8122338dca45 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:54.865430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a89f92-43c2-4f68-8ed4-8122338dca45 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:54.865724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a89f92-43c2-4f68-8ed4-8122338dca45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:54.866526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a89f92-43c2-4f68-8ed4-8122338dca45 None admin] RBAC: Adding query filter params (role.id=c7b4bfc82f3b46cfaa10773dad7cd03a, scope.project.id=a1dea3ca8e8f4df987f62c98bb0ffe26, user.id=ecaf2631bf924cbd9bb4a2cd9d812c17) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:10:54.866839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a89f92-43c2-4f68-8ed4-8122338dca45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:54.868296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a89f92-43c2-4f68-8ed4-8122338dca45 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:54.874453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 548/1117] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:10:54 2022] GET /identity/v3/role_assignments?user.id=ecaf2631bf924cbd9bb4a2cd9d812c17&scope.project.id=a1dea3ca8e8f4df987f62c98bb0ffe26&role.id=c7b4bfc82f3b46cfaa10773dad7cd03a => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:56.829237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca8369-d176-4278-a4fc-4f3f1ce56cfe None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:56.829810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca8369-d176-4278-a4fc-4f3f1ce56cfe None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:56.829810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca8369-d176-4278-a4fc-4f3f1ce56cfe None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:56.831171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 570/1118] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:56.836610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589497-290e-48d2-9b98-79ee9bb4fdf9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:56.837049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589497-290e-48d2-9b98-79ee9bb4fdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:56.837151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589497-290e-48d2-9b98-79ee9bb4fdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:56.859722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2e589497-290e-48d2-9b98-79ee9bb4fdf9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:56.861558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2e589497-290e-48d2-9b98-79ee9bb4fdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:56.935058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 549/1119] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:56.958803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca8369-d176-4278-a4fc-4f3f1ce56cfe None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:57.024570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:57.058527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca8369-d176-4278-a4fc-4f3f1ce56cfe None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:57.092706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f334cb-c1a1-4f29-8f56-658b4e1b66d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:57.093353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f334cb-c1a1-4f29-8f56-658b4e1b66d2 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:57.093558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f334cb-c1a1-4f29-8f56-658b4e1b66d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:57.093842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f334cb-c1a1-4f29-8f56-658b4e1b66d2 None admin] PATH_INFO: `/v3/roles/c7b4bfc82f3b46cfaa10773dad7cd03a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:57.095337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f334cb-c1a1-4f29-8f56-658b4e1b66d2 None admin] RBAC: Authorizing `identity:get_role(role_id=c7b4bfc82f3b46cfaa10773dad7cd03a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:57.096708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f334cb-c1a1-4f29-8f56-658b4e1b66d2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:57.098104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 571/1120] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:10:56 2022] GET /identity/v3/roles/c7b4bfc82f3b46cfaa10773dad7cd03a => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:57.103918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e589497-290e-48d2-9b98-79ee9bb4fdf9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:57.200897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e589497-290e-48d2-9b98-79ee9bb4fdf9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:57.221784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:57.235726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae8d9a-97e1-4df3-bb3c-3cd1807f5fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:57.236274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae8d9a-97e1-4df3-bb3c-3cd1807f5fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:57.236562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae8d9a-97e1-4df3-bb3c-3cd1807f5fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:57.236842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae8d9a-97e1-4df3-bb3c-3cd1807f5fc1 None admin] PATH_INFO: `/v3/users/ecaf2631bf924cbd9bb4a2cd9d812c17` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:57.238044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae8d9a-97e1-4df3-bb3c-3cd1807f5fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=ecaf2631bf924cbd9bb4a2cd9d812c17)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:57.239503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae8d9a-97e1-4df3-bb3c-3cd1807f5fc1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:57.241133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 550/1121] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:10:57 2022] GET /identity/v3/users/ecaf2631bf924cbd9bb4a2cd9d812c17 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:57.247167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f334cb-c1a1-4f29-8f56-658b4e1b66d2 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:57.351064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f334cb-c1a1-4f29-8f56-658b4e1b66d2 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:57.389090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bda40-d27e-4902-9ac0-75d2eeea4ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:57.389765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bda40-d27e-4902-9ac0-75d2eeea4ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:57.390052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bda40-d27e-4902-9ac0-75d2eeea4ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:57.390331 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bda40-d27e-4902-9ac0-75d2eeea4ed4 None admin] PATH_INFO: `/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:57.391497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bda40-d27e-4902-9ac0-75d2eeea4ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=a1dea3ca8e8f4df987f62c98bb0ffe26)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:57.392918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bda40-d27e-4902-9ac0-75d2eeea4ed4 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:57.394627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 572/1122] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:10:57 2022] GET /identity/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:57.400636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae8d9a-97e1-4df3-bb3c-3cd1807f5fc1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:57.500596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae8d9a-97e1-4df3-bb3c-3cd1807f5fc1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:57.533705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38385350-f117-4fe9-96c1-eefdc0c8d167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:57.534374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385350-f117-4fe9-96c1-eefdc0c8d167 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:57.534658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385350-f117-4fe9-96c1-eefdc0c8d167 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:57.534938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385350-f117-4fe9-96c1-eefdc0c8d167 None admin] PATH_INFO: `/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26/users/ecaf2631bf924cbd9bb4a2cd9d812c17/roles/c7b4bfc82f3b46cfaa10773dad7cd03a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:57.537103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38385350-f117-4fe9-96c1-eefdc0c8d167 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1dea3ca8e8f4df987f62c98bb0ffe26, user_id=ecaf2631bf924cbd9bb4a2cd9d812c17, role_id=c7b4bfc82f3b46cfaa10773dad7cd03a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:57.538592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38385350-f117-4fe9-96c1-eefdc0c8d167 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:57.620843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 551/1123] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:10:57 2022] PUT /identity/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26/users/ecaf2631bf924cbd9bb4a2cd9d812c17/roles/c7b4bfc82f3b46cfaa10773dad7cd03a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:10:59.709287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a39b99-59da-40e6-bb31-6035a9c1ea09 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:59.709824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a39b99-59da-40e6-bb31-6035a9c1ea09 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:59.709824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a39b99-59da-40e6-bb31-6035a9c1ea09 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:59.711175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 573/1124] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:10:59.716298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d4a69-8e5c-425f-8e30-e329237ef0c3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:59.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d4a69-8e5c-425f-8e30-e329237ef0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:59.716869 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d4a69-8e5c-425f-8e30-e329237ef0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:59.735581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d97d4a69-8e5c-425f-8e30-e329237ef0c3 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:10:59.736748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d97d4a69-8e5c-425f-8e30-e329237ef0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:10:59.779152 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:59.801610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 552/1125] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:10:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:10:59.826025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a39b99-59da-40e6-bb31-6035a9c1ea09 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:10:59.829085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:10:59.932285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a39b99-59da-40e6-bb31-6035a9c1ea09 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:10:59.966021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7febb3-7334-4792-9ce0-5c1aa3b7bcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:10:59.966584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7febb3-7334-4792-9ce0-5c1aa3b7bcea None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:10:59.967087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7febb3-7334-4792-9ce0-5c1aa3b7bcea None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:10:59.967377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7febb3-7334-4792-9ce0-5c1aa3b7bcea None admin] PATH_INFO: `/v3/roles/c7b4bfc82f3b46cfaa10773dad7cd03a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:10:59.969017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7febb3-7334-4792-9ce0-5c1aa3b7bcea None admin] RBAC: Authorizing `identity:get_role(role_id=c7b4bfc82f3b46cfaa10773dad7cd03a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:10:59.970426 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7febb3-7334-4792-9ce0-5c1aa3b7bcea None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:10:59.971776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 574/1126] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:10:59 2022] GET /identity/v3/roles/c7b4bfc82f3b46cfaa10773dad7cd03a => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:10:59.977670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d4a69-8e5c-425f-8e30-e329237ef0c3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:00.074636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d4a69-8e5c-425f-8e30-e329237ef0c3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:00.108052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f8e1f-cbbc-4c0d-8d8e-15024bd6741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:00.108695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f8e1f-cbbc-4c0d-8d8e-15024bd6741e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:00.108986 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f8e1f-cbbc-4c0d-8d8e-15024bd6741e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:00.109263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f8e1f-cbbc-4c0d-8d8e-15024bd6741e None admin] PATH_INFO: `/v3/users/ecaf2631bf924cbd9bb4a2cd9d812c17` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:00.110455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f8e1f-cbbc-4c0d-8d8e-15024bd6741e None admin] RBAC: Authorizing `identity:get_user(user_id=ecaf2631bf924cbd9bb4a2cd9d812c17)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:00.111906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f8e1f-cbbc-4c0d-8d8e-15024bd6741e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:00.113566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 553/1127] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:10:59 2022] GET /identity/v3/users/ecaf2631bf924cbd9bb4a2cd9d812c17 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:00.119437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7febb3-7334-4792-9ce0-5c1aa3b7bcea None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:00.188527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:00.219335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7febb3-7334-4792-9ce0-5c1aa3b7bcea None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:00.253327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a1149-c9c8-443f-9565-d79d8a6369f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:00.253955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a1149-c9c8-443f-9565-d79d8a6369f9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:00.254239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a1149-c9c8-443f-9565-d79d8a6369f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:00.254517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a1149-c9c8-443f-9565-d79d8a6369f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:00.320860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a1149-c9c8-443f-9565-d79d8a6369f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:00.322340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a1149-c9c8-443f-9565-d79d8a6369f9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:00.389697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:00.422697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 575/1128] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:11:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:11:00.427793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f8e1f-cbbc-4c0d-8d8e-15024bd6741e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:00.474047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:00.526979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f8e1f-cbbc-4c0d-8d8e-15024bd6741e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:00.560402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10877c5-57a9-42be-842e-358387d41b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:00.561036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10877c5-57a9-42be-842e-358387d41b0d None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:00.561348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10877c5-57a9-42be-842e-358387d41b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:00.561630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10877c5-57a9-42be-842e-358387d41b0d None admin] PATH_INFO: `/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:00.562791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10877c5-57a9-42be-842e-358387d41b0d None admin] RBAC: Authorizing `identity:get_project(project_id=a1dea3ca8e8f4df987f62c98bb0ffe26)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:00.564231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10877c5-57a9-42be-842e-358387d41b0d None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:00.565931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 554/1129] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:11:00 2022] GET /identity/v3/projects/a1dea3ca8e8f4df987f62c98bb0ffe26 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:00.572210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a1149-c9c8-443f-9565-d79d8a6369f9 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:00.669242 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a1149-c9c8-443f-9565-d79d8a6369f9 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:00.703366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b991dc-0a18-4512-9a09-c6a73068f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:00.703962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b991dc-0a18-4512-9a09-c6a73068f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:00.704249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b991dc-0a18-4512-9a09-c6a73068f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:00.704527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b991dc-0a18-4512-9a09-c6a73068f4c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:00.705390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b991dc-0a18-4512-9a09-c6a73068f4c4 None admin] RBAC: Adding query filter params (role.id=c7b4bfc82f3b46cfaa10773dad7cd03a, scope.project.id=a1dea3ca8e8f4df987f62c98bb0ffe26, user.id=ecaf2631bf924cbd9bb4a2cd9d812c17) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:00.705655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b991dc-0a18-4512-9a09-c6a73068f4c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:00.707081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b991dc-0a18-4512-9a09-c6a73068f4c4 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:00.712793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 576/1130] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:11:00 2022] GET /identity/v3/role_assignments?user.id=ecaf2631bf924cbd9bb4a2cd9d812c17&scope.project.id=a1dea3ca8e8f4df987f62c98bb0ffe26&role.id=c7b4bfc82f3b46cfaa10773dad7cd03a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:02.735424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adfff4-bc14-433a-9221-14906cdd22ed None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:02.735947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adfff4-bc14-433a-9221-14906cdd22ed None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:02.735947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adfff4-bc14-433a-9221-14906cdd22ed None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:02.737231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 555/1131] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:11:02.742510 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedb599-d7ba-4f53-b46a-e755c2a02fae None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:02.742933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedb599-d7ba-4f53-b46a-e755c2a02fae None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:02.743093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedb599-d7ba-4f53-b46a-e755c2a02fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:02.761905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-fdedb599-d7ba-4f53-b46a-e755c2a02fae None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:02.763107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-fdedb599-d7ba-4f53-b46a-e755c2a02fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:02.789169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:02.829224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 577/1132] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:11:02.851903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21adfff4-bc14-433a-9221-14906cdd22ed None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:02.962461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21adfff4-bc14-433a-9221-14906cdd22ed None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:02.976673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:02.996513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8da1e-8f32-43c4-ba33-59c3e901f24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:02.997071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8da1e-8f32-43c4-ba33-59c3e901f24f None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:02.997389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8da1e-8f32-43c4-ba33-59c3e901f24f None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:02.997713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8da1e-8f32-43c4-ba33-59c3e901f24f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:02.998993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8da1e-8f32-43c4-ba33-59c3e901f24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:03.000441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8da1e-8f32-43c4-ba33-59c3e901f24f None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:03.002069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 556/1133] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:11:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:03.007935 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedb599-d7ba-4f53-b46a-e755c2a02fae None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:03.105227 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedb599-d7ba-4f53-b46a-e755c2a02fae None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:03.139885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7db16b-d772-4e82-85c4-f53a9437c416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:03.140516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7db16b-d772-4e82-85c4-f53a9437c416 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:03.140806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7db16b-d772-4e82-85c4-f53a9437c416 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:03.141091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7db16b-d772-4e82-85c4-f53a9437c416 None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:03.141832 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7db16b-d772-4e82-85c4-f53a9437c416 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:03.143389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7db16b-d772-4e82-85c4-f53a9437c416 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:03.273950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 578/1134] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:11:03 2022] POST /identity/v3/projects => generated 311 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:11:05.348999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cb03c-2817-4dbe-9c19-50173538a3b1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:05.349569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cb03c-2817-4dbe-9c19-50173538a3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:05.349569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cb03c-2817-4dbe-9c19-50173538a3b1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:05.350941 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 557/1135] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:11:05.356129 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2df5c-e46d-45a7-92e5-5006d7f7a199 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:05.356557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2df5c-e46d-45a7-92e5-5006d7f7a199 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:05.356704 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2df5c-e46d-45a7-92e5-5006d7f7a199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:05.376054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-57f2df5c-e46d-45a7-92e5-5006d7f7a199 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:05.377259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-57f2df5c-e46d-45a7-92e5-5006d7f7a199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:05.388919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:05.445146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 579/1136] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:11:05.470255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cb03c-2817-4dbe-9c19-50173538a3b1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:05.572109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cb03c-2817-4dbe-9c19-50173538a3b1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:05.606596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b331ba-56d4-4fa9-be9a-4eb4190abe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:05.607377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b331ba-56d4-4fa9-be9a-4eb4190abe59 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:05.607598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b331ba-56d4-4fa9-be9a-4eb4190abe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:05.607917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b331ba-56d4-4fa9-be9a-4eb4190abe59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:05.609074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b331ba-56d4-4fa9-be9a-4eb4190abe59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:05.610591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b331ba-56d4-4fa9-be9a-4eb4190abe59 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:05.612225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 558/1137] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:11:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:05.618200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2df5c-e46d-45a7-92e5-5006d7f7a199 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:05.714620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2df5c-e46d-45a7-92e5-5006d7f7a199 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:05.748737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb4c53-84e5-4e30-a06d-5271897fd2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:05.749380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb4c53-84e5-4e30-a06d-5271897fd2e2 None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:05.749673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb4c53-84e5-4e30-a06d-5271897fd2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:05.749987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb4c53-84e5-4e30-a06d-5271897fd2e2 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:05.750659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb4c53-84e5-4e30-a06d-5271897fd2e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:05.752100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb4c53-84e5-4e30-a06d-5271897fd2e2 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:05.823878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 580/1138] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:11:05 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:11:07.875114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b046e67-d35f-437c-a05b-d2765ae15bad None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:07.875675 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b046e67-d35f-437c-a05b-d2765ae15bad None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:07.875675 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b046e67-d35f-437c-a05b-d2765ae15bad None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:07.877045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 559/1139] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:11:07.882239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e5b1-74c3-45fe-b1d0-a2684b7cb959 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:07.882685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e5b1-74c3-45fe-b1d0-a2684b7cb959 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:07.882806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e5b1-74c3-45fe-b1d0-a2684b7cb959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:07.890813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:07.902806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-1456e5b1-74c3-45fe-b1d0-a2684b7cb959 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:07.904631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1456e5b1-74c3-45fe-b1d0-a2684b7cb959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:07.972635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 581/1140] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:11:07.994909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b046e67-d35f-437c-a05b-d2765ae15bad None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:08.046247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:08.095978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b046e67-d35f-437c-a05b-d2765ae15bad None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:08.128938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a594073-d038-41c5-b33f-0b094f46725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:08.129530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a594073-d038-41c5-b33f-0b094f46725b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:08.129820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a594073-d038-41c5-b33f-0b094f46725b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:08.130109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a594073-d038-41c5-b33f-0b094f46725b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:08.140966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a594073-d038-41c5-b33f-0b094f46725b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8a594073-d038-41c5-b33f-0b094f46725b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:11:08.145829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:11:08.146922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:11:08.147829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 560/1141] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:11:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:11:08.153065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456e5b1-74c3-45fe-b1d0-a2684b7cb959 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:08.250727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456e5b1-74c3-45fe-b1d0-a2684b7cb959 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:08.280433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:08.285289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c764e8-9c45-4f56-87c9-a789200845cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:08.285943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c764e8-9c45-4f56-87c9-a789200845cc None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:08.286231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c764e8-9c45-4f56-87c9-a789200845cc None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:08.286511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c764e8-9c45-4f56-87c9-a789200845cc None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:08.287236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c764e8-9c45-4f56-87c9-a789200845cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:08.287547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c764e8-9c45-4f56-87c9-a789200845cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:08.289020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c764e8-9c45-4f56-87c9-a789200845cc None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:08.297271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 582/1142] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:11:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:08.303212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a594073-d038-41c5-b33f-0b094f46725b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:08.405744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a594073-d038-41c5-b33f-0b094f46725b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:08.417374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:08.440859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd68cb-43df-4a22-905c-1d27b37f6ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:08.441524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd68cb-43df-4a22-905c-1d27b37f6ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:08.441817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd68cb-43df-4a22-905c-1d27b37f6ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:08.442160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd68cb-43df-4a22-905c-1d27b37f6ab9 None admin] PATH_INFO: `/v3/users/69df4b76129f4e61b8b50ced37804849` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:08.455994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd68cb-43df-4a22-905c-1d27b37f6ab9 None admin] RBAC: Authorizing `identity:get_user(user_id=69df4b76129f4e61b8b50ced37804849)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:08.457459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd68cb-43df-4a22-905c-1d27b37f6ab9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:08.459089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 561/1143] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:11:08 2022] GET /identity/v3/users/69df4b76129f4e61b8b50ced37804849 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:08.465014 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c764e8-9c45-4f56-87c9-a789200845cc None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:08.565923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c764e8-9c45-4f56-87c9-a789200845cc None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:08.599526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb88a1-1020-486c-b29f-1b2fcd99ecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:08.600144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb88a1-1020-486c-b29f-1b2fcd99ecdb None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:08.600432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb88a1-1020-486c-b29f-1b2fcd99ecdb None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:08.600710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb88a1-1020-486c-b29f-1b2fcd99ecdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:08.662040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:08.669217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb88a1-1020-486c-b29f-1b2fcd99ecdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:08.670732 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb88a1-1020-486c-b29f-1b2fcd99ecdb None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:08.768393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 583/1144] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:11:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:11:08.774282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd68cb-43df-4a22-905c-1d27b37f6ab9 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:08.871500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd68cb-43df-4a22-905c-1d27b37f6ab9 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:08.905955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30772d11-b690-4aa6-8e49-de543d08fe99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:08.906588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30772d11-b690-4aa6-8e49-de543d08fe99 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:08.906853 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30772d11-b690-4aa6-8e49-de543d08fe99 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:08.907124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30772d11-b690-4aa6-8e49-de543d08fe99 None admin] PATH_INFO: `/v3/projects/e916f2840b40454c97cce16379de36db` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:08.908263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30772d11-b690-4aa6-8e49-de543d08fe99 None admin] RBAC: Authorizing `identity:get_project(project_id=e916f2840b40454c97cce16379de36db)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:08.909730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30772d11-b690-4aa6-8e49-de543d08fe99 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:08.911413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 562/1145] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:11:08 2022] GET /identity/v3/projects/e916f2840b40454c97cce16379de36db => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:08.917800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb88a1-1020-486c-b29f-1b2fcd99ecdb None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:09.024434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:09.031993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb88a1-1020-486c-b29f-1b2fcd99ecdb None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:09.066263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b02295-9d03-4c8b-8a9b-d9567497d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:09.066870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b02295-9d03-4c8b-8a9b-d9567497d53b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:09.067153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b02295-9d03-4c8b-8a9b-d9567497d53b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:09.067428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b02295-9d03-4c8b-8a9b-d9567497d53b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:09.068215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b02295-9d03-4c8b-8a9b-d9567497d53b None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=e916f2840b40454c97cce16379de36db, user.id=69df4b76129f4e61b8b50ced37804849) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:09.068524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b02295-9d03-4c8b-8a9b-d9567497d53b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:09.069981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b02295-9d03-4c8b-8a9b-d9567497d53b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:09.075236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 584/1146] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:11:08 2022] GET /identity/v3/role_assignments?user.id=69df4b76129f4e61b8b50ced37804849&scope.project.id=e916f2840b40454c97cce16379de36db&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:11.054535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093413e1-b98f-43ff-9558-26896efa980e None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:11.055143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093413e1-b98f-43ff-9558-26896efa980e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:11.055143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093413e1-b98f-43ff-9558-26896efa980e None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:11.056431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 563/1147] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:11:11.062298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d7c1c-e587-435e-b977-6addf798eb48 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:11.062980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d7c1c-e587-435e-b977-6addf798eb48 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:11.063233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d7c1c-e587-435e-b977-6addf798eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:11.081168 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-8e3d7c1c-e587-435e-b977-6addf798eb48 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:11.082401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8e3d7c1c-e587-435e-b977-6addf798eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:11.147392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 585/1148] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:11:11.170540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093413e1-b98f-43ff-9558-26896efa980e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:11.205718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:11.272063 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093413e1-b98f-43ff-9558-26896efa980e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:11.304922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762f892-3f7c-4722-a02f-e1bb943f98d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:11.305521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f892-3f7c-4722-a02f-e1bb943f98d6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:11.305815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f892-3f7c-4722-a02f-e1bb943f98d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:11.306097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f892-3f7c-4722-a02f-e1bb943f98d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:11.316876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762f892-3f7c-4722-a02f-e1bb943f98d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6762f892-3f7c-4722-a02f-e1bb943f98d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:11:11.321822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:11:11.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:11:11.324156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 564/1149] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:11:11.329659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d7c1c-e587-435e-b977-6addf798eb48 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:11.415846 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:11.427600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d7c1c-e587-435e-b977-6addf798eb48 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:11.462746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25b0e-b7b4-462e-9ef4-98aba198f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:11.463359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25b0e-b7b4-462e-9ef4-98aba198f706 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:11.463646 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25b0e-b7b4-462e-9ef4-98aba198f706 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:11.463922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25b0e-b7b4-462e-9ef4-98aba198f706 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:11.464635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed25b0e-b7b4-462e-9ef4-98aba198f706 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:11.464941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed25b0e-b7b4-462e-9ef4-98aba198f706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:11.466568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed25b0e-b7b4-462e-9ef4-98aba198f706 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:11.474568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 586/1150] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:11:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:11.480599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762f892-3f7c-4722-a02f-e1bb943f98d6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:11.577526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:11.583254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762f892-3f7c-4722-a02f-e1bb943f98d6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:11.616560 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5852291-7128-4a46-bf8a-b6f1ea1d3994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:11.617196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5852291-7128-4a46-bf8a-b6f1ea1d3994 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:11.617504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5852291-7128-4a46-bf8a-b6f1ea1d3994 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:11.617781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5852291-7128-4a46-bf8a-b6f1ea1d3994 None admin] PATH_INFO: `/v3/users/69df4b76129f4e61b8b50ced37804849` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:11.618931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5852291-7128-4a46-bf8a-b6f1ea1d3994 None admin] RBAC: Authorizing `identity:get_user(user_id=69df4b76129f4e61b8b50ced37804849)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:11.620366 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5852291-7128-4a46-bf8a-b6f1ea1d3994 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:11.622039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 565/1151] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:11:11 2022] GET /identity/v3/users/69df4b76129f4e61b8b50ced37804849 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:11:11.628012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25b0e-b7b4-462e-9ef4-98aba198f706 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:11.726155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25b0e-b7b4-462e-9ef4-98aba198f706 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:11.758423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd666d63-2d07-4484-8437-3dbbdedfe3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:11.759049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd666d63-2d07-4484-8437-3dbbdedfe3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:11.759328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd666d63-2d07-4484-8437-3dbbdedfe3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:11.759605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd666d63-2d07-4484-8437-3dbbdedfe3e9 None admin] PATH_INFO: `/v3/projects/e916f2840b40454c97cce16379de36db` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:11.760745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd666d63-2d07-4484-8437-3dbbdedfe3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e916f2840b40454c97cce16379de36db)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:11.762197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd666d63-2d07-4484-8437-3dbbdedfe3e9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:11.764626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 587/1152] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:11:11 2022] GET /identity/v3/projects/e916f2840b40454c97cce16379de36db => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:11.770632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5852291-7128-4a46-bf8a-b6f1ea1d3994 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:11.869208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5852291-7128-4a46-bf8a-b6f1ea1d3994 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:11.903287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037de03b-73a6-4457-b746-6d2f9d3d6b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:11.904834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037de03b-73a6-4457-b746-6d2f9d3d6b0c None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:11.905120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037de03b-73a6-4457-b746-6d2f9d3d6b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:11.905426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037de03b-73a6-4457-b746-6d2f9d3d6b0c None admin] PATH_INFO: `/v3/projects/e916f2840b40454c97cce16379de36db/users/69df4b76129f4e61b8b50ced37804849/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:11.907367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037de03b-73a6-4457-b746-6d2f9d3d6b0c None admin] RBAC: Authorizing `identity:create_grant(project_id=e916f2840b40454c97cce16379de36db, user_id=69df4b76129f4e61b8b50ced37804849, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:11.908808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037de03b-73a6-4457-b746-6d2f9d3d6b0c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:12.016227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 566/1153] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:11:11 2022] PUT /identity/v3/projects/e916f2840b40454c97cce16379de36db/users/69df4b76129f4e61b8b50ced37804849/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:11:14.019115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00bbce-e09e-4d2d-8104-e80c5d20db45 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:14.019652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00bbce-e09e-4d2d-8104-e80c5d20db45 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:14.019740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00bbce-e09e-4d2d-8104-e80c5d20db45 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:14.021041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 588/1154] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:11:14.026456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e2564-6f37-4911-a963-b770b988fe23 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:14.026862 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e2564-6f37-4911-a963-b770b988fe23 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:14.027015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e2564-6f37-4911-a963-b770b988fe23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:14.045505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-be2e2564-6f37-4911-a963-b770b988fe23 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:14.046676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-be2e2564-6f37-4911-a963-b770b988fe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:14.063508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:14.113015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 567/1155] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:11:14.136180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00bbce-e09e-4d2d-8104-e80c5d20db45 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:14.155901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:14.237901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00bbce-e09e-4d2d-8104-e80c5d20db45 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:14.271546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26289476-6bf3-433b-a555-29a45302f242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:14.272138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26289476-6bf3-433b-a555-29a45302f242 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:14.272423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26289476-6bf3-433b-a555-29a45302f242 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:14.272701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26289476-6bf3-433b-a555-29a45302f242 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:14.283251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26289476-6bf3-433b-a555-29a45302f242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-26289476-6bf3-433b-a555-29a45302f242 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:11:14.288051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:11:14.289173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:11:14.290348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 589/1156] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:11:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:11:14.295472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e2564-6f37-4911-a963-b770b988fe23 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:14.398694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e2564-6f37-4911-a963-b770b988fe23 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:14.432230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98547e5-c03e-4ef0-bdc2-ed80c33cf758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:14.432846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98547e5-c03e-4ef0-bdc2-ed80c33cf758 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:14.433133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98547e5-c03e-4ef0-bdc2-ed80c33cf758 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:14.433447 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98547e5-c03e-4ef0-bdc2-ed80c33cf758 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:14.434168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98547e5-c03e-4ef0-bdc2-ed80c33cf758 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:14.435004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98547e5-c03e-4ef0-bdc2-ed80c33cf758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:14.436378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98547e5-c03e-4ef0-bdc2-ed80c33cf758 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:14.444618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 568/1157] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:11:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:14.450979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26289476-6bf3-433b-a555-29a45302f242 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:14.529514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:14.550814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26289476-6bf3-433b-a555-29a45302f242 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:14.584853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fcad4-8794-490a-a938-ff4b6724cf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:14.585522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fcad4-8794-490a-a938-ff4b6724cf85 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:14.585814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fcad4-8794-490a-a938-ff4b6724cf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:14.586097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fcad4-8794-490a-a938-ff4b6724cf85 None admin] PATH_INFO: `/v3/users/69df4b76129f4e61b8b50ced37804849` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:14.587257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fcad4-8794-490a-a938-ff4b6724cf85 None admin] RBAC: Authorizing `identity:get_user(user_id=69df4b76129f4e61b8b50ced37804849)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:14.588697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fcad4-8794-490a-a938-ff4b6724cf85 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:14.590345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 590/1158] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:11:14 2022] GET /identity/v3/users/69df4b76129f4e61b8b50ced37804849 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:14.596999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98547e5-c03e-4ef0-bdc2-ed80c33cf758 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:14.615314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:14.695059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98547e5-c03e-4ef0-bdc2-ed80c33cf758 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:14.728377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195272b-2a85-4d3a-986a-ec647ecfe419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:14.728991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195272b-2a85-4d3a-986a-ec647ecfe419 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:14.729272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195272b-2a85-4d3a-986a-ec647ecfe419 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:14.729575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195272b-2a85-4d3a-986a-ec647ecfe419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:14.796036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195272b-2a85-4d3a-986a-ec647ecfe419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:14.797530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195272b-2a85-4d3a-986a-ec647ecfe419 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:14.814857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:14.896776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 569/1159] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:11:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:11:14.902707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fcad4-8794-490a-a938-ff4b6724cf85 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:15.010928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fcad4-8794-490a-a938-ff4b6724cf85 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:15.049561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b435a-4630-416f-8ab3-c023af36fc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:15.050216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b435a-4630-416f-8ab3-c023af36fc9e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:15.050399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b435a-4630-416f-8ab3-c023af36fc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:15.050675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b435a-4630-416f-8ab3-c023af36fc9e None admin] PATH_INFO: `/v3/projects/e916f2840b40454c97cce16379de36db` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:15.051823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b435a-4630-416f-8ab3-c023af36fc9e None admin] RBAC: Authorizing `identity:get_project(project_id=e916f2840b40454c97cce16379de36db)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:15.053242 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b435a-4630-416f-8ab3-c023af36fc9e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:15.054936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 591/1160] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:11:14 2022] GET /identity/v3/projects/e916f2840b40454c97cce16379de36db => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:15.061057 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195272b-2a85-4d3a-986a-ec647ecfe419 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:15.159119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195272b-2a85-4d3a-986a-ec647ecfe419 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:15.175933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:15.193008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d6853b-da71-433d-8b1b-bde41e4483e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:15.193658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6853b-da71-433d-8b1b-bde41e4483e8 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:15.193942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6853b-da71-433d-8b1b-bde41e4483e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:15.194223 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6853b-da71-433d-8b1b-bde41e4483e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:15.195073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d6853b-da71-433d-8b1b-bde41e4483e8 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=e916f2840b40454c97cce16379de36db, user.id=69df4b76129f4e61b8b50ced37804849) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:15.195392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d6853b-da71-433d-8b1b-bde41e4483e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:15.196841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d6853b-da71-433d-8b1b-bde41e4483e8 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:15.202711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 570/1161] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:11:15 2022] GET /identity/v3/role_assignments?user.id=69df4b76129f4e61b8b50ced37804849&scope.project.id=e916f2840b40454c97cce16379de36db&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:17.232440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94eb9b-161b-4d21-bed6-9bfd5cd07808 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:17.233016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94eb9b-161b-4d21-bed6-9bfd5cd07808 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:17.233016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94eb9b-161b-4d21-bed6-9bfd5cd07808 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:17.234402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 592/1162] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:11:17.239858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53324ffc-2b1a-4074-9ee0-053b600e22ae None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:17.240319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53324ffc-2b1a-4074-9ee0-053b600e22ae None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:17.240414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53324ffc-2b1a-4074-9ee0-053b600e22ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:17.259301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-53324ffc-2b1a-4074-9ee0-053b600e22ae None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:17.260476 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-53324ffc-2b1a-4074-9ee0-053b600e22ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:17.335364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 571/1163] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:11:17.370898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94eb9b-161b-4d21-bed6-9bfd5cd07808 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:17.389682 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:17.473941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94eb9b-161b-4d21-bed6-9bfd5cd07808 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:17.506443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a33d4-0ab0-4f44-b953-95d91bc65586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:17.507056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a33d4-0ab0-4f44-b953-95d91bc65586 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:17.507337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a33d4-0ab0-4f44-b953-95d91bc65586 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:17.507873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a33d4-0ab0-4f44-b953-95d91bc65586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:17.572994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a33d4-0ab0-4f44-b953-95d91bc65586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:17.574464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a33d4-0ab0-4f44-b953-95d91bc65586 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:17.588828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:17.670183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 593/1164] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:11:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:11:17.677014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53324ffc-2b1a-4074-9ee0-053b600e22ae None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:17.774113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53324ffc-2b1a-4074-9ee0-053b600e22ae None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:17.785083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:17.807834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6fdb0-540f-4ad3-b9a6-a12272381f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:17.808797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6fdb0-540f-4ad3-b9a6-a12272381f71 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:17.808797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6fdb0-540f-4ad3-b9a6-a12272381f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:17.809086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6fdb0-540f-4ad3-b9a6-a12272381f71 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:17.816006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6fdb0-540f-4ad3-b9a6-a12272381f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:17.817822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6fdb0-540f-4ad3-b9a6-a12272381f71 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-8cd6fdb0-540f-4ad3-b9a6-a12272381f71 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:11:17.829700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:11:17.830912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:11:17.831795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 572/1165] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 28 06:11:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:11:17.837118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a33d4-0ab0-4f44-b953-95d91bc65586 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:17.935910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a33d4-0ab0-4f44-b953-95d91bc65586 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:17.950833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:17.970339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a1fb4-90af-4c80-b9ba-a5e063725061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:17.970961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1fb4-90af-4c80-b9ba-a5e063725061 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:17.971244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1fb4-90af-4c80-b9ba-a5e063725061 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:17.971519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1fb4-90af-4c80-b9ba-a5e063725061 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:17.972211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a1fb4-90af-4c80-b9ba-a5e063725061 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:17.972518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a1fb4-90af-4c80-b9ba-a5e063725061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:17.973924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a1fb4-90af-4c80-b9ba-a5e063725061 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:17.981143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 594/1166] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 28 06:11:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:19.998009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cd4b9-99a5-4293-babb-107f3e11aa33 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:19.998648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cd4b9-99a5-4293-babb-107f3e11aa33 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:19.998648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cd4b9-99a5-4293-babb-107f3e11aa33 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:19.999951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 573/1167] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:11:20.005039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68154998-6a5c-41a1-b072-fd6a0b7d6dc1 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:20.005526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68154998-6a5c-41a1-b072-fd6a0b7d6dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:20.005626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68154998-6a5c-41a1-b072-fd6a0b7d6dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:20.023752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-68154998-6a5c-41a1-b072-fd6a0b7d6dc1 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:20.024935 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-68154998-6a5c-41a1-b072-fd6a0b7d6dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:20.094310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 595/1168] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:11:20.117043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cd4b9-99a5-4293-babb-107f3e11aa33 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:20.218001 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cd4b9-99a5-4293-babb-107f3e11aa33 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:20.251126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a0dc5-cb2e-49e9-9adc-bb52ab4912e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:20.251674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a0dc5-cb2e-49e9-9adc-bb52ab4912e7 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:20.251964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a0dc5-cb2e-49e9-9adc-bb52ab4912e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:20.252246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a0dc5-cb2e-49e9-9adc-bb52ab4912e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:20.252971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a0dc5-cb2e-49e9-9adc-bb52ab4912e7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:20.254384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a0dc5-cb2e-49e9-9adc-bb52ab4912e7 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:20.330295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 574/1169] 10.222.0.69 () {60 vars in 1138 bytes} [Thu Jul 28 06:11:20 2022] POST /identity/v3/domains => generated 267 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:11:22.384831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd7495-0186-4c9d-bb6d-ef8b598a278c None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:22.385394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd7495-0186-4c9d-bb6d-ef8b598a278c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:22.385514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd7495-0186-4c9d-bb6d-ef8b598a278c None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:22.386829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 596/1170] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:11:22.392689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f496b71-a8c8-40d7-91c4-06db184584df None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:22.393134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f496b71-a8c8-40d7-91c4-06db184584df None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:22.393260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f496b71-a8c8-40d7-91c4-06db184584df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:22.412476 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8f496b71-a8c8-40d7-91c4-06db184584df None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:22.413728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8f496b71-a8c8-40d7-91c4-06db184584df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:22.439419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:22.482481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 575/1171] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:11:22.505927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd7495-0186-4c9d-bb6d-ef8b598a278c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:22.607667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd7495-0186-4c9d-bb6d-ef8b598a278c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:22.617712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:22.644123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdea00-b853-46cd-82f6-e51d6a22d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:22.644773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdea00-b853-46cd-82f6-e51d6a22d48e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:22.645059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdea00-b853-46cd-82f6-e51d6a22d48e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:22.645361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdea00-b853-46cd-82f6-e51d6a22d48e None admin] PATH_INFO: `/v3/domains/5a3425f6adf24c97b8518e74e657fd0a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:22.656461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdea00-b853-46cd-82f6-e51d6a22d48e None admin] RBAC: Authorizing `identity:get_domain(domain_id=5a3425f6adf24c97b8518e74e657fd0a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:22.657958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdea00-b853-46cd-82f6-e51d6a22d48e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:22.659658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 597/1172] 10.222.0.69 () {56 vars in 1183 bytes} [Thu Jul 28 06:11:22 2022] GET /identity/v3/domains/5a3425f6adf24c97b8518e74e657fd0a => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:11:22.665610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f496b71-a8c8-40d7-91c4-06db184584df None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:22.772534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f496b71-a8c8-40d7-91c4-06db184584df None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:22.806273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681a6e1-4838-4f27-9bdc-8a05e0fafd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:22.806885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a6e1-4838-4f27-9bdc-8a05e0fafd05 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:22.807175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a6e1-4838-4f27-9bdc-8a05e0fafd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:22.807453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a6e1-4838-4f27-9bdc-8a05e0fafd05 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:22.808133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681a6e1-4838-4f27-9bdc-8a05e0fafd05 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:22.810328 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681a6e1-4838-4f27-9bdc-8a05e0fafd05 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:22.922231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 576/1173] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:11:22 2022] POST /identity/v3/projects => generated 361 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:11:24.999258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbfcaf-2abe-4d71-98b7-0abde678b43e None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:24.999831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbfcaf-2abe-4d71-98b7-0abde678b43e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:24.999831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbfcaf-2abe-4d71-98b7-0abde678b43e None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:25.001177 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 598/1174] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:11:25.006412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd16627-3d7c-4aa5-9bff-55dd2ef16662 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:25.006837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd16627-3d7c-4aa5-9bff-55dd2ef16662 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:25.006957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd16627-3d7c-4aa5-9bff-55dd2ef16662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:25.030811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6fd16627-3d7c-4aa5-9bff-55dd2ef16662 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:25.032666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6fd16627-3d7c-4aa5-9bff-55dd2ef16662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:25.040522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:25.106516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 577/1175] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:11:25.130917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbfcaf-2abe-4d71-98b7-0abde678b43e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:25.231520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbfcaf-2abe-4d71-98b7-0abde678b43e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:25.265371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e35566-0b86-4ed9-b8df-759bde6461e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:25.265946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35566-0b86-4ed9-b8df-759bde6461e9 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:25.266335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35566-0b86-4ed9-b8df-759bde6461e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:25.266624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35566-0b86-4ed9-b8df-759bde6461e9 None admin] PATH_INFO: `/v3/domains/5a3425f6adf24c97b8518e74e657fd0a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:25.267789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e35566-0b86-4ed9-b8df-759bde6461e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5a3425f6adf24c97b8518e74e657fd0a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:25.269239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e35566-0b86-4ed9-b8df-759bde6461e9 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:25.270876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 599/1176] 10.222.0.69 () {56 vars in 1183 bytes} [Thu Jul 28 06:11:25 2022] GET /identity/v3/domains/5a3425f6adf24c97b8518e74e657fd0a => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:25.276697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd16627-3d7c-4aa5-9bff-55dd2ef16662 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:25.376180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd16627-3d7c-4aa5-9bff-55dd2ef16662 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:25.413319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ab765-af1c-4293-9281-50715c956f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:25.413972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ab765-af1c-4293-9281-50715c956f9e None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:25.414257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ab765-af1c-4293-9281-50715c956f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:25.414536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ab765-af1c-4293-9281-50715c956f9e None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:25.415230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ab765-af1c-4293-9281-50715c956f9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:25.416680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ab765-af1c-4293-9281-50715c956f9e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:25.511630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:25.518899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 578/1177] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:11:25 2022] POST /identity/v3/users => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:11:27.577665 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c459cb0-9e6d-4f41-83b2-2dd8064fb137 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:27.578220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c459cb0-9e6d-4f41-83b2-2dd8064fb137 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:27.578220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c459cb0-9e6d-4f41-83b2-2dd8064fb137 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:27.579591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 600/1178] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:11:27.584940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b69719-613c-4323-a5bc-43498a7e80fd None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:27.585410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b69719-613c-4323-a5bc-43498a7e80fd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:27.585506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b69719-613c-4323-a5bc-43498a7e80fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:27.603685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-25b69719-613c-4323-a5bc-43498a7e80fd None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:27.604859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-25b69719-613c-4323-a5bc-43498a7e80fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:27.669091 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 579/1179] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:11:27.692113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c459cb0-9e6d-4f41-83b2-2dd8064fb137 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:27.728021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:27.793733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c459cb0-9e6d-4f41-83b2-2dd8064fb137 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:27.827651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45960681-b9fc-4c8d-a85b-e69e9d941dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:27.828284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45960681-b9fc-4c8d-a85b-e69e9d941dba None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:27.828572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45960681-b9fc-4c8d-a85b-e69e9d941dba None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:27.828852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45960681-b9fc-4c8d-a85b-e69e9d941dba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:27.839064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45960681-b9fc-4c8d-a85b-e69e9d941dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-45960681-b9fc-4c8d-a85b-e69e9d941dba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:11:27.844378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:11:27.845574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:11:27.846508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 601/1180] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:11:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:11:27.851782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b69719-613c-4323-a5bc-43498a7e80fd None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:27.948480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b69719-613c-4323-a5bc-43498a7e80fd None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:27.974212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:27.982084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9852bd-608d-40df-945a-74850158222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:27.982698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9852bd-608d-40df-945a-74850158222b None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:27.992087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9852bd-608d-40df-945a-74850158222b None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:27.992087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9852bd-608d-40df-945a-74850158222b None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:27.992087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9852bd-608d-40df-945a-74850158222b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:27.992087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9852bd-608d-40df-945a-74850158222b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:27.992087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9852bd-608d-40df-945a-74850158222b None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:27.993392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 580/1181] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:11:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:27.999416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45960681-b9fc-4c8d-a85b-e69e9d941dba None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:28.092060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:28.097865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45960681-b9fc-4c8d-a85b-e69e9d941dba None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:28.130203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81185c8e-2799-4f16-bd7e-8d546a32fbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:28.130829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81185c8e-2799-4f16-bd7e-8d546a32fbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:28.131114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81185c8e-2799-4f16-bd7e-8d546a32fbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:28.131391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81185c8e-2799-4f16-bd7e-8d546a32fbf1 None admin] PATH_INFO: `/v3/users/c3d11f623c3f4e7c80eb8f3f052db19f` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:28.146729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81185c8e-2799-4f16-bd7e-8d546a32fbf1 None admin] RBAC: Authorizing `identity:get_user(user_id=c3d11f623c3f4e7c80eb8f3f052db19f)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:28.148165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81185c8e-2799-4f16-bd7e-8d546a32fbf1 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:28.149818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 602/1182] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:11:27 2022] GET /identity/v3/users/c3d11f623c3f4e7c80eb8f3f052db19f => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:28.156249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9852bd-608d-40df-945a-74850158222b None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:28.252656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9852bd-608d-40df-945a-74850158222b None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:28.286756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbbf82-5ebc-4e5e-b409-c845ccfad147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:28.287367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbbf82-5ebc-4e5e-b409-c845ccfad147 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:28.287655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbbf82-5ebc-4e5e-b409-c845ccfad147 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:28.287936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbbf82-5ebc-4e5e-b409-c845ccfad147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:28.344364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:28.354295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbbf82-5ebc-4e5e-b409-c845ccfad147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:28.363404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbbf82-5ebc-4e5e-b409-c845ccfad147 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:28.456500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 581/1183] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:11:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:11:28.462406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81185c8e-2799-4f16-bd7e-8d546a32fbf1 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:28.561546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81185c8e-2799-4f16-bd7e-8d546a32fbf1 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:28.594657 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c4e27-26d0-4501-a383-a244d683bd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:28.595309 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c4e27-26d0-4501-a383-a244d683bd38 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:28.595588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c4e27-26d0-4501-a383-a244d683bd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:28.595867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c4e27-26d0-4501-a383-a244d683bd38 None admin] PATH_INFO: `/v3/projects/0761aad0f5204beaa4bd89a69a8a036c` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:28.597043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c4e27-26d0-4501-a383-a244d683bd38 None admin] RBAC: Authorizing `identity:get_project(project_id=0761aad0f5204beaa4bd89a69a8a036c)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:28.598524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c4e27-26d0-4501-a383-a244d683bd38 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:28.600236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 603/1184] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:11:28 2022] GET /identity/v3/projects/0761aad0f5204beaa4bd89a69a8a036c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:28.606848 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbbf82-5ebc-4e5e-b409-c845ccfad147 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:28.696111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:28.706036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbbf82-5ebc-4e5e-b409-c845ccfad147 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:28.739146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e37810-e3d2-46d9-b241-db48ad38c930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:28.739758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e37810-e3d2-46d9-b241-db48ad38c930 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:28.740048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e37810-e3d2-46d9-b241-db48ad38c930 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:28.740328 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e37810-e3d2-46d9-b241-db48ad38c930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:28.741121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e37810-e3d2-46d9-b241-db48ad38c930 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=0761aad0f5204beaa4bd89a69a8a036c, user.id=c3d11f623c3f4e7c80eb8f3f052db19f) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:28.741466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e37810-e3d2-46d9-b241-db48ad38c930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:28.742913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e37810-e3d2-46d9-b241-db48ad38c930 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:28.748865 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 582/1185] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:11:28 2022] GET /identity/v3/role_assignments?user.id=c3d11f623c3f4e7c80eb8f3f052db19f&scope.project.id=0761aad0f5204beaa4bd89a69a8a036c&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:30.730649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4df864-170a-4321-98b5-5d9f842d3c63 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:30.731243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4df864-170a-4321-98b5-5d9f842d3c63 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:30.731243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4df864-170a-4321-98b5-5d9f842d3c63 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:30.732520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 604/1186] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:11:30.738379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dfbd9-cba7-4477-bfad-6f00dac98e21 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:30.738793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dfbd9-cba7-4477-bfad-6f00dac98e21 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:30.738935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dfbd9-cba7-4477-bfad-6f00dac98e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:30.761852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-261dfbd9-cba7-4477-bfad-6f00dac98e21 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:30.763071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-261dfbd9-cba7-4477-bfad-6f00dac98e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:30.831053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 583/1187] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:11:30.853507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4df864-170a-4321-98b5-5d9f842d3c63 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:30.878253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:30.954706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4df864-170a-4321-98b5-5d9f842d3c63 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:30.988109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a72ace-57be-4be3-a27b-7ca89b35869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:30.988680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a72ace-57be-4be3-a27b-7ca89b35869f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:30.988965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a72ace-57be-4be3-a27b-7ca89b35869f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:30.989244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a72ace-57be-4be3-a27b-7ca89b35869f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:30.999604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a72ace-57be-4be3-a27b-7ca89b35869f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-54a72ace-57be-4be3-a27b-7ca89b35869f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:11:31.004370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:11:31.005482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:11:31.006567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 605/1188] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:11:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:11:31.012768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dfbd9-cba7-4477-bfad-6f00dac98e21 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:31.099932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:31.116410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dfbd9-cba7-4477-bfad-6f00dac98e21 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:31.150601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932105a5-a5c6-4d52-8d79-8fb291f0a7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:31.151215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932105a5-a5c6-4d52-8d79-8fb291f0a7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:31.151503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932105a5-a5c6-4d52-8d79-8fb291f0a7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:31.151791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932105a5-a5c6-4d52-8d79-8fb291f0a7e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:31.152538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932105a5-a5c6-4d52-8d79-8fb291f0a7e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:31.152844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932105a5-a5c6-4d52-8d79-8fb291f0a7e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:31.154253 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932105a5-a5c6-4d52-8d79-8fb291f0a7e4 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:31.162236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 584/1189] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:11:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:31.168538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a72ace-57be-4be3-a27b-7ca89b35869f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:31.248914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:31.266707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a72ace-57be-4be3-a27b-7ca89b35869f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:31.300894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd3167-81c1-47ef-93fa-4e76db874505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:31.301561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd3167-81c1-47ef-93fa-4e76db874505 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:31.301853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd3167-81c1-47ef-93fa-4e76db874505 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:31.302134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd3167-81c1-47ef-93fa-4e76db874505 None admin] PATH_INFO: `/v3/users/c3d11f623c3f4e7c80eb8f3f052db19f` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:31.303304 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd3167-81c1-47ef-93fa-4e76db874505 None admin] RBAC: Authorizing `identity:get_user(user_id=c3d11f623c3f4e7c80eb8f3f052db19f)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:31.304740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd3167-81c1-47ef-93fa-4e76db874505 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:31.306379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 606/1190] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:11:31 2022] GET /identity/v3/users/c3d11f623c3f4e7c80eb8f3f052db19f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:31.312905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932105a5-a5c6-4d52-8d79-8fb291f0a7e4 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:31.414065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932105a5-a5c6-4d52-8d79-8fb291f0a7e4 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:31.447968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305c2dd-6268-49dc-9959-31772d5d311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:31.448620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305c2dd-6268-49dc-9959-31772d5d311a None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:31.448910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305c2dd-6268-49dc-9959-31772d5d311a None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:31.449256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305c2dd-6268-49dc-9959-31772d5d311a None admin] PATH_INFO: `/v3/projects/0761aad0f5204beaa4bd89a69a8a036c` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:31.450502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305c2dd-6268-49dc-9959-31772d5d311a None admin] RBAC: Authorizing `identity:get_project(project_id=0761aad0f5204beaa4bd89a69a8a036c)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:31.451951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305c2dd-6268-49dc-9959-31772d5d311a None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:31.453658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 585/1191] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:11:31 2022] GET /identity/v3/projects/0761aad0f5204beaa4bd89a69a8a036c => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:31.459979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd3167-81c1-47ef-93fa-4e76db874505 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:31.560957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd3167-81c1-47ef-93fa-4e76db874505 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:31.596074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3f84-2269-4a49-b070-d186c1531f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:31.596747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3f84-2269-4a49-b070-d186c1531f82 None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:31.597037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3f84-2269-4a49-b070-d186c1531f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:31.597349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3f84-2269-4a49-b070-d186c1531f82 None admin] PATH_INFO: `/v3/projects/0761aad0f5204beaa4bd89a69a8a036c/users/c3d11f623c3f4e7c80eb8f3f052db19f/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:31.599289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3f84-2269-4a49-b070-d186c1531f82 None admin] RBAC: Authorizing `identity:create_grant(project_id=0761aad0f5204beaa4bd89a69a8a036c, user_id=c3d11f623c3f4e7c80eb8f3f052db19f, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:31.600735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3f84-2269-4a49-b070-d186c1531f82 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:31.673418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 607/1192] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:11:31 2022] PUT /identity/v3/projects/0761aad0f5204beaa4bd89a69a8a036c/users/c3d11f623c3f4e7c80eb8f3f052db19f/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:11:33.682581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ce3f5-0be5-4558-8c79-0f2a807b2929 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:33.683116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ce3f5-0be5-4558-8c79-0f2a807b2929 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:33.683116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ce3f5-0be5-4558-8c79-0f2a807b2929 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:33.684603 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 586/1193] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:11:33.689786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd99f8-8e0c-4bc7-a28a-958a78eb8272 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:33.690219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd99f8-8e0c-4bc7-a28a-958a78eb8272 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:33.690327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd99f8-8e0c-4bc7-a28a-958a78eb8272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:33.708171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-c4cd99f8-8e0c-4bc7-a28a-958a78eb8272 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:11:33.709393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c4cd99f8-8e0c-4bc7-a28a-958a78eb8272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:11:33.712143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:33.774655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 608/1194] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:11:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:11:33.798992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ce3f5-0be5-4558-8c79-0f2a807b2929 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:33.815571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:33.903417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ce3f5-0be5-4558-8c79-0f2a807b2929 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:33.936494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fde843-edae-4ff8-b780-1aae903868a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:33.937051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fde843-edae-4ff8-b780-1aae903868a3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:33.937351 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fde843-edae-4ff8-b780-1aae903868a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:33.937630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fde843-edae-4ff8-b780-1aae903868a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:33.947863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fde843-edae-4ff8-b780-1aae903868a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e0fde843-edae-4ff8-b780-1aae903868a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:11:33.953512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:11:33.954669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:11:33.955899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 587/1195] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:11:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:11:33.960763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd99f8-8e0c-4bc7-a28a-958a78eb8272 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:34.057989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd99f8-8e0c-4bc7-a28a-958a78eb8272 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:34.091424 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73ce0b-0af1-4b51-95e9-a9ae2d0795ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:34.092043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73ce0b-0af1-4b51-95e9-a9ae2d0795ae None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:34.092328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73ce0b-0af1-4b51-95e9-a9ae2d0795ae None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:34.092607 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73ce0b-0af1-4b51-95e9-a9ae2d0795ae None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:34.093342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73ce0b-0af1-4b51-95e9-a9ae2d0795ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:34.093654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73ce0b-0af1-4b51-95e9-a9ae2d0795ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:34.095025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73ce0b-0af1-4b51-95e9-a9ae2d0795ae None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:34.102878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 609/1196] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:11:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:34.109273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fde843-edae-4ff8-b780-1aae903868a3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:34.186179 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:34.209855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fde843-edae-4ff8-b780-1aae903868a3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:34.243770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802c03b-3153-4a25-81dc-668150ed3a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:34.244409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802c03b-3153-4a25-81dc-668150ed3a88 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:34.244695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802c03b-3153-4a25-81dc-668150ed3a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:34.244970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802c03b-3153-4a25-81dc-668150ed3a88 None admin] PATH_INFO: `/v3/users/c3d11f623c3f4e7c80eb8f3f052db19f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:34.246884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802c03b-3153-4a25-81dc-668150ed3a88 None admin] RBAC: Authorizing `identity:get_user(user_id=c3d11f623c3f4e7c80eb8f3f052db19f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:34.248326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802c03b-3153-4a25-81dc-668150ed3a88 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:34.249959 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 588/1197] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 28 06:11:34 2022] GET /identity/v3/users/c3d11f623c3f4e7c80eb8f3f052db19f => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:34.256045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73ce0b-0af1-4b51-95e9-a9ae2d0795ae None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:34.259197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:34.354118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73ce0b-0af1-4b51-95e9-a9ae2d0795ae None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:34.396277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04774673-d7c5-47e1-82c1-416688fd4a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:34.396963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04774673-d7c5-47e1-82c1-416688fd4a71 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:34.397260 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04774673-d7c5-47e1-82c1-416688fd4a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:34.397573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04774673-d7c5-47e1-82c1-416688fd4a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:34.466116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04774673-d7c5-47e1-82c1-416688fd4a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:34.467496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04774673-d7c5-47e1-82c1-416688fd4a71 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:34.469370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:34.564701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 610/1198] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:11:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 06:11:34.570690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802c03b-3153-4a25-81dc-668150ed3a88 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:34.671054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802c03b-3153-4a25-81dc-668150ed3a88 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:34.703507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f9961-ca7c-444f-942d-863f04816b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:34.704128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f9961-ca7c-444f-942d-863f04816b69 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:34.704411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f9961-ca7c-444f-942d-863f04816b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:34.704689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f9961-ca7c-444f-942d-863f04816b69 None admin] PATH_INFO: `/v3/projects/0761aad0f5204beaa4bd89a69a8a036c` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:34.705873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f9961-ca7c-444f-942d-863f04816b69 None admin] RBAC: Authorizing `identity:get_project(project_id=0761aad0f5204beaa4bd89a69a8a036c)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:34.707314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f9961-ca7c-444f-942d-863f04816b69 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:34.708975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 589/1199] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 28 06:11:34 2022] GET /identity/v3/projects/0761aad0f5204beaa4bd89a69a8a036c => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:34.716009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04774673-d7c5-47e1-82c1-416688fd4a71 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:11:34.813845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04774673-d7c5-47e1-82c1-416688fd4a71 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:11:34.815383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:34.849461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3eeffb-4f6e-469f-b75a-55323e2355ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:11:34.850086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eeffb-4f6e-469f-b75a-55323e2355ae None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:11:34.850375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eeffb-4f6e-469f-b75a-55323e2355ae None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:11:34.850654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eeffb-4f6e-469f-b75a-55323e2355ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:11:34.851456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3eeffb-4f6e-469f-b75a-55323e2355ae None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=0761aad0f5204beaa4bd89a69a8a036c, user.id=c3d11f623c3f4e7c80eb8f3f052db19f) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:11:34.851770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3eeffb-4f6e-469f-b75a-55323e2355ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:11:34.853211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3eeffb-4f6e-469f-b75a-55323e2355ae None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:11:34.859178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 611/1200] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:11:34 2022] GET /identity/v3/role_assignments?user.id=c3d11f623c3f4e7c80eb8f3f052db19f&scope.project.id=0761aad0f5204beaa4bd89a69a8a036c&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:11:44.699212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:11:44.856552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=17 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:47.023231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfefdea-45b5-4f2a-8be6-da96f5e69155 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:47.023831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfefdea-45b5-4f2a-8be6-da96f5e69155 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:47.023831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfefdea-45b5-4f2a-8be6-da96f5e69155 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:47.025143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 590/1201] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:12:47.030551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92790e4e-593e-4a0b-9886-a2eb7a4d4ef8 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:47.031030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92790e4e-593e-4a0b-9886-a2eb7a4d4ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:47.031131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92790e4e-593e-4a0b-9886-a2eb7a4d4ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:47.049181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-92790e4e-593e-4a0b-9886-a2eb7a4d4ef8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:12:47.050443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-92790e4e-593e-4a0b-9886-a2eb7a4d4ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:12:47.116827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 612/1202] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:12:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:12:47.142121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfefdea-45b5-4f2a-8be6-da96f5e69155 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:47.262961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfefdea-45b5-4f2a-8be6-da96f5e69155 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:47.302327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde452ae-da44-40b1-9ad7-2c7d41d10670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:47.302327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde452ae-da44-40b1-9ad7-2c7d41d10670 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:47.302327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde452ae-da44-40b1-9ad7-2c7d41d10670 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:47.302327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde452ae-da44-40b1-9ad7-2c7d41d10670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:47.309772 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde452ae-da44-40b1-9ad7-2c7d41d10670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:47.311227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde452ae-da44-40b1-9ad7-2c7d41d10670 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-dde452ae-da44-40b1-9ad7-2c7d41d10670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:12:47.325548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:47.326616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:47.327599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:12:47.327599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:12:47.327599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:47.327599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:12:47.327599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 591/1203] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:47.332777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92790e4e-593e-4a0b-9886-a2eb7a4d4ef8 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:47.439336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92790e4e-593e-4a0b-9886-a2eb7a4d4ef8 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:47.475061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:47.482079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8103e1-ed7f-4907-984f-9ddb7d9f5511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:47.482713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8103e1-ed7f-4907-984f-9ddb7d9f5511 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:47.483003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8103e1-ed7f-4907-984f-9ddb7d9f5511 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:47.483289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8103e1-ed7f-4907-984f-9ddb7d9f5511 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:47.483996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8103e1-ed7f-4907-984f-9ddb7d9f5511 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:47.484366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8103e1-ed7f-4907-984f-9ddb7d9f5511 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:47.485887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8103e1-ed7f-4907-984f-9ddb7d9f5511 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:47.497280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 613/1204] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:47.503625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde452ae-da44-40b1-9ad7-2c7d41d10670 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:47.558376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:47.605644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde452ae-da44-40b1-9ad7-2c7d41d10670 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:47.639082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd3024-c301-45a2-b8e8-94559f542059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:47.639819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd3024-c301-45a2-b8e8-94559f542059 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:47.640106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd3024-c301-45a2-b8e8-94559f542059 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:47.641052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd3024-c301-45a2-b8e8-94559f542059 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:47.641786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd3024-c301-45a2-b8e8-94559f542059 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:47.643284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd3024-c301-45a2-b8e8-94559f542059 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:47.714629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 592/1205] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 28 06:12:47 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:12:49.820198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faece33a-3303-41dc-a302-3592e7ae0d17 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:49.820782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faece33a-3303-41dc-a302-3592e7ae0d17 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:49.821073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faece33a-3303-41dc-a302-3592e7ae0d17 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:49.823228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 614/1206] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:12:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:12:49.828585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa0514-7b08-477a-a060-c0f6305d2549 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:49.830454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa0514-7b08-477a-a060-c0f6305d2549 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:49.830454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa0514-7b08-477a-a060-c0f6305d2549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:49.848627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fbfa0514-7b08-477a-a060-c0f6305d2549 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:12:49.853425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fbfa0514-7b08-477a-a060-c0f6305d2549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:12:49.918255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 593/1207] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:12:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:12:49.945589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faece33a-3303-41dc-a302-3592e7ae0d17 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:50.049923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faece33a-3303-41dc-a302-3592e7ae0d17 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:50.082324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ce38d-a2b8-42f8-9b15-ab55c44c139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:50.082897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ce38d-a2b8-42f8-9b15-ab55c44c139f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:50.083182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ce38d-a2b8-42f8-9b15-ab55c44c139f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:50.083493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ce38d-a2b8-42f8-9b15-ab55c44c139f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:50.094484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ce38d-a2b8-42f8-9b15-ab55c44c139f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-f11ce38d-a2b8-42f8-9b15-ab55c44c139f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:12:50.099406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:50.100632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:12:50.102022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 615/1208] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:12:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:50.107523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa0514-7b08-477a-a060-c0f6305d2549 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:50.158723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:50.222667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa0514-7b08-477a-a060-c0f6305d2549 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:50.257528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c90c7-117f-4c4d-8370-13c0cc5c7574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:50.258152 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c90c7-117f-4c4d-8370-13c0cc5c7574 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:50.258441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c90c7-117f-4c4d-8370-13c0cc5c7574 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:50.258724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c90c7-117f-4c4d-8370-13c0cc5c7574 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:50.259444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c90c7-117f-4c4d-8370-13c0cc5c7574 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:50.259756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c90c7-117f-4c4d-8370-13c0cc5c7574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:50.261193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c90c7-117f-4c4d-8370-13c0cc5c7574 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:50.269773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 594/1209] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:12:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:50.275563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ce38d-a2b8-42f8-9b15-ab55c44c139f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:50.327100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:50.378296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ce38d-a2b8-42f8-9b15-ab55c44c139f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:50.413881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d501a-4d64-4318-ba7b-92f4a5018216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:50.414526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d501a-4d64-4318-ba7b-92f4a5018216 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:50.414812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d501a-4d64-4318-ba7b-92f4a5018216 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:50.415093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d501a-4d64-4318-ba7b-92f4a5018216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:50.425908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d501a-4d64-4318-ba7b-92f4a5018216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:50.427399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d501a-4d64-4318-ba7b-92f4a5018216 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-988d501a-4d64-4318-ba7b-92f4a5018216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:12:50.441355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:50.444376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:50.445501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:12:50.445501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:12:50.445501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:50.445501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:12:50.445501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 616/1210] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:50.448300 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c90c7-117f-4c4d-8370-13c0cc5c7574 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:50.551273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:50.554371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c90c7-117f-4c4d-8370-13c0cc5c7574 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:50.587499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04948492-f173-4ddc-a464-bc44df017315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:50.588116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04948492-f173-4ddc-a464-bc44df017315 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:50.588405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04948492-f173-4ddc-a464-bc44df017315 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:50.588687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04948492-f173-4ddc-a464-bc44df017315 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:50.589411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04948492-f173-4ddc-a464-bc44df017315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:50.589725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04948492-f173-4ddc-a464-bc44df017315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:50.591890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04948492-f173-4ddc-a464-bc44df017315 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:50.604225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 595/1211] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:50.610046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d501a-4d64-4318-ba7b-92f4a5018216 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:50.710469 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d501a-4d64-4318-ba7b-92f4a5018216 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:50.711918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:50.745015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e085cf-1efc-4575-b6e9-91d220664310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:50.745690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e085cf-1efc-4575-b6e9-91d220664310 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:50.745978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e085cf-1efc-4575-b6e9-91d220664310 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:50.746256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e085cf-1efc-4575-b6e9-91d220664310 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:50.754851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e085cf-1efc-4575-b6e9-91d220664310 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:50.756294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e085cf-1efc-4575-b6e9-91d220664310 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-65e085cf-1efc-4575-b6e9-91d220664310 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:12:50.767370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:50.768508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:12:50.769736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 617/1212] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:12:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:50.774587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04948492-f173-4ddc-a464-bc44df017315 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:50.875619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04948492-f173-4ddc-a464-bc44df017315 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:50.910298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5419a-88a7-48f7-8f5a-ad571d88d581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:50.910917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5419a-88a7-48f7-8f5a-ad571d88d581 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:50.911195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5419a-88a7-48f7-8f5a-ad571d88d581 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:50.911467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5419a-88a7-48f7-8f5a-ad571d88d581 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:50.912254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5419a-88a7-48f7-8f5a-ad571d88d581 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:50.912579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5419a-88a7-48f7-8f5a-ad571d88d581 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:50.914128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5419a-88a7-48f7-8f5a-ad571d88d581 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:50.931959 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 596/1213] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 28 06:12:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:50.938058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e085cf-1efc-4575-b6e9-91d220664310 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:51.039408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e085cf-1efc-4575-b6e9-91d220664310 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:51.072544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f81a5e-fb37-4813-9ec2-d2a59633d670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:51.073208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81a5e-fb37-4813-9ec2-d2a59633d670 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:51.073526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81a5e-fb37-4813-9ec2-d2a59633d670 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:51.073810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81a5e-fb37-4813-9ec2-d2a59633d670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:51.104924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:51.143100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f81a5e-fb37-4813-9ec2-d2a59633d670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:51.144575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f81a5e-fb37-4813-9ec2-d2a59633d670 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:51.243123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 618/1214] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:12:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:12:51.250143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5419a-88a7-48f7-8f5a-ad571d88d581 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:51.269267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:51.351327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5419a-88a7-48f7-8f5a-ad571d88d581 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:51.387660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c0723-202c-44ae-a441-90c38826d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:51.388316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c0723-202c-44ae-a441-90c38826d3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:51.388607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c0723-202c-44ae-a441-90c38826d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:51.388890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c0723-202c-44ae-a441-90c38826d3e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:51.399236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c0723-202c-44ae-a441-90c38826d3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:51.400706 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c0723-202c-44ae-a441-90c38826d3e3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-640c0723-202c-44ae-a441-90c38826d3e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:12:51.415883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:51.416979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:51.418084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:12:51.418084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:12:51.418084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:51.418084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:12:51.418084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 597/1215] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:51.423594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f81a5e-fb37-4813-9ec2-d2a59633d670 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:51.485785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:51.536093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f81a5e-fb37-4813-9ec2-d2a59633d670 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:51.572466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84a105-aac8-4c3e-b368-a6af1d7fddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:51.573091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84a105-aac8-4c3e-b368-a6af1d7fddb3 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:51.573418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84a105-aac8-4c3e-b368-a6af1d7fddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:51.573696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84a105-aac8-4c3e-b368-a6af1d7fddb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:51.574442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84a105-aac8-4c3e-b368-a6af1d7fddb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:51.574723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84a105-aac8-4c3e-b368-a6af1d7fddb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:51.576896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84a105-aac8-4c3e-b368-a6af1d7fddb3 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:51.588402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 619/1216] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:51.595242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c0723-202c-44ae-a441-90c38826d3e3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:51.666254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:51.697080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c0723-202c-44ae-a441-90c38826d3e3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:51.733327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b460690-7907-41b5-acee-3f375c3dafe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:51.733972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460690-7907-41b5-acee-3f375c3dafe2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:51.734263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460690-7907-41b5-acee-3f375c3dafe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:51.734546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460690-7907-41b5-acee-3f375c3dafe2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:51.741941 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b460690-7907-41b5-acee-3f375c3dafe2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:51.743401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b460690-7907-41b5-acee-3f375c3dafe2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-9b460690-7907-41b5-acee-3f375c3dafe2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:12:51.755130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:51.756203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:51.757383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:12:51.757383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:12:51.757383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:12:51.757383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:12:51.757383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:12:51.757383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:12:51.757383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:12:51.757383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:12:51.757383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 598/1217] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:51.762545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84a105-aac8-4c3e-b368-a6af1d7fddb3 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:51.866753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84a105-aac8-4c3e-b368-a6af1d7fddb3 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:51.890064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:51.901462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6c65c-0433-407b-a1ef-a03b00b5b160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:51.902088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6c65c-0433-407b-a1ef-a03b00b5b160 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:51.902378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6c65c-0433-407b-a1ef-a03b00b5b160 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:51.902662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6c65c-0433-407b-a1ef-a03b00b5b160 None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:51.904140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6c65c-0433-407b-a1ef-a03b00b5b160 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:51.904459 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6c65c-0433-407b-a1ef-a03b00b5b160 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:51.905926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6c65c-0433-407b-a1ef-a03b00b5b160 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:51.917751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 620/1218] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:12:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:51.924017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b460690-7907-41b5-acee-3f375c3dafe2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:52.027251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b460690-7907-41b5-acee-3f375c3dafe2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:52.055699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:52.062038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324d1dd-3830-494a-a3c0-942b1d3b146e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:52.062657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324d1dd-3830-494a-a3c0-942b1d3b146e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:52.062946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324d1dd-3830-494a-a3c0-942b1d3b146e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:52.063228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324d1dd-3830-494a-a3c0-942b1d3b146e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:52.064020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324d1dd-3830-494a-a3c0-942b1d3b146e None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=244a8787df7641f2bac9b51213ddb322) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:52.064337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324d1dd-3830-494a-a3c0-942b1d3b146e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:52.065820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324d1dd-3830-494a-a3c0-942b1d3b146e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:52.071100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 599/1219] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:12:51 2022] GET /identity/v3/role_assignments?user.id=244a8787df7641f2bac9b51213ddb322&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:54.136580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3bcb4-f02a-45bc-8e03-8be6b42aeb1b None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:54.137185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3bcb4-f02a-45bc-8e03-8be6b42aeb1b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:54.137185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3bcb4-f02a-45bc-8e03-8be6b42aeb1b None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:54.138649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 621/1220] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:12:54.144298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a031787-3782-4821-8516-20488310665d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:54.144785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a031787-3782-4821-8516-20488310665d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:54.144883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a031787-3782-4821-8516-20488310665d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:54.163570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7a031787-3782-4821-8516-20488310665d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:12:54.164796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7a031787-3782-4821-8516-20488310665d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:12:54.232565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 600/1221] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:12:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:12:54.255471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3bcb4-f02a-45bc-8e03-8be6b42aeb1b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:54.359377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3bcb4-f02a-45bc-8e03-8be6b42aeb1b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:54.404272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a982a5-85dc-4797-aa69-82106fe04145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:54.404842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a982a5-85dc-4797-aa69-82106fe04145 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:54.405126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a982a5-85dc-4797-aa69-82106fe04145 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:54.405436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a982a5-85dc-4797-aa69-82106fe04145 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:54.415662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a982a5-85dc-4797-aa69-82106fe04145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-64a982a5-85dc-4797-aa69-82106fe04145 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:12:54.421124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:54.422388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:12:54.423390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 622/1222] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:12:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:54.428736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a031787-3782-4821-8516-20488310665d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:54.530961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a031787-3782-4821-8516-20488310665d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:54.555352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:54.565916 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333efdf4-7e02-459c-85e8-eabec27a2469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:54.566538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333efdf4-7e02-459c-85e8-eabec27a2469 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:54.566824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333efdf4-7e02-459c-85e8-eabec27a2469 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:54.567102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333efdf4-7e02-459c-85e8-eabec27a2469 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:54.567824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333efdf4-7e02-459c-85e8-eabec27a2469 None admin] RBAC: Adding query filter params (name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:54.568134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333efdf4-7e02-459c-85e8-eabec27a2469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:54.569561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333efdf4-7e02-459c-85e8-eabec27a2469 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:54.577767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 601/1223] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:12:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:54.583679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a982a5-85dc-4797-aa69-82106fe04145 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:54.629147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:54.685535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a982a5-85dc-4797-aa69-82106fe04145 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:54.720237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b86b1-b073-46c0-80f1-9d828a5abc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:54.720873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b86b1-b073-46c0-80f1-9d828a5abc76 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:54.721154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b86b1-b073-46c0-80f1-9d828a5abc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:54.721461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b86b1-b073-46c0-80f1-9d828a5abc76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:54.731954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b86b1-b073-46c0-80f1-9d828a5abc76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:54.733428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b86b1-b073-46c0-80f1-9d828a5abc76 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-776b86b1-b073-46c0-80f1-9d828a5abc76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:12:54.747985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:54.749109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:54.750245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:12:54.750245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:12:54.750245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:54.750245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:12:54.750245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 623/1224] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:54.755413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333efdf4-7e02-459c-85e8-eabec27a2469 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:54.857334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333efdf4-7e02-459c-85e8-eabec27a2469 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:54.892877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4f706-0337-4af3-840f-d00ad02312f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:54.893194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4f706-0337-4af3-840f-d00ad02312f6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:54.893518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4f706-0337-4af3-840f-d00ad02312f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:54.893802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4f706-0337-4af3-840f-d00ad02312f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:54.894497 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4f706-0337-4af3-840f-d00ad02312f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:54.894808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4f706-0337-4af3-840f-d00ad02312f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:54.896205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4f706-0337-4af3-840f-d00ad02312f6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:54.909116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 602/1225] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:54.915194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b86b1-b073-46c0-80f1-9d828a5abc76 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:55.009562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:55.016958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b86b1-b073-46c0-80f1-9d828a5abc76 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:55.050406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f8cdc-3c58-4ec6-b427-36110691823a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:55.051053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f8cdc-3c58-4ec6-b427-36110691823a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:55.051344 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f8cdc-3c58-4ec6-b427-36110691823a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:55.051627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f8cdc-3c58-4ec6-b427-36110691823a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:55.058936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f8cdc-3c58-4ec6-b427-36110691823a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:55.060401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f8cdc-3c58-4ec6-b427-36110691823a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-a23f8cdc-3c58-4ec6-b427-36110691823a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:12:55.071389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:55.072515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:12:55.073629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 624/1226] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:12:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:55.079192 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4f706-0337-4af3-840f-d00ad02312f6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:55.127520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:55.179511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4f706-0337-4af3-840f-d00ad02312f6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:55.214763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d51b58-f3e3-4e37-9420-a9b7a04528b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:55.215503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d51b58-f3e3-4e37-9420-a9b7a04528b3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:55.215782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d51b58-f3e3-4e37-9420-a9b7a04528b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:55.216072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d51b58-f3e3-4e37-9420-a9b7a04528b3 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:55.216854 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d51b58-f3e3-4e37-9420-a9b7a04528b3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:55.217159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d51b58-f3e3-4e37-9420-a9b7a04528b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:55.218670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d51b58-f3e3-4e37-9420-a9b7a04528b3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:55.235850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 603/1227] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 28 06:12:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:55.241778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f8cdc-3c58-4ec6-b427-36110691823a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:55.342776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f8cdc-3c58-4ec6-b427-36110691823a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:55.378136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c2b5d-32c6-4f75-9dc3-eb50f93e6e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:55.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2b5d-32c6-4f75-9dc3-eb50f93e6e1b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:55.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2b5d-32c6-4f75-9dc3-eb50f93e6e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:55.379476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2b5d-32c6-4f75-9dc3-eb50f93e6e1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:55.390192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c2b5d-32c6-4f75-9dc3-eb50f93e6e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:55.391685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c2b5d-32c6-4f75-9dc3-eb50f93e6e1b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-827c2b5d-32c6-4f75-9dc3-eb50f93e6e1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:12:55.406662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:55.407845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:55.408949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:12:55.408949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:12:55.408949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:55.408949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:12:55.408949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 625/1228] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:55.413841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d51b58-f3e3-4e37-9420-a9b7a04528b3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:55.517631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d51b58-f3e3-4e37-9420-a9b7a04528b3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:55.519939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:55.554787 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5ffe-7e42-468a-908d-ae951345e9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:55.555401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5ffe-7e42-468a-908d-ae951345e9df None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:55.555685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5ffe-7e42-468a-908d-ae951345e9df None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:55.555962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5ffe-7e42-468a-908d-ae951345e9df None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:55.556655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe5ffe-7e42-468a-908d-ae951345e9df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:55.556962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe5ffe-7e42-468a-908d-ae951345e9df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:55.558468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe5ffe-7e42-468a-908d-ae951345e9df None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:55.570294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 604/1229] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:55.576176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c2b5d-32c6-4f75-9dc3-eb50f93e6e1b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:55.585499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:55.675149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c2b5d-32c6-4f75-9dc3-eb50f93e6e1b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:55.708338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60b2e8-338e-4f0d-b482-6b617c000397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:55.709224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60b2e8-338e-4f0d-b482-6b617c000397 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:55.709548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60b2e8-338e-4f0d-b482-6b617c000397 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:55.710591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60b2e8-338e-4f0d-b482-6b617c000397 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:55.717827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60b2e8-338e-4f0d-b482-6b617c000397 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:55.719341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60b2e8-338e-4f0d-b482-6b617c000397 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-0b60b2e8-338e-4f0d-b482-6b617c000397 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:12:55.730437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:55.731569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:55.732697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:12:55.732697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:12:55.732697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:12:55.732697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:12:55.732697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:12:55.732697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:12:55.732697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:12:55.732697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:12:55.732697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 626/1230] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:12:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:12:55.737716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5ffe-7e42-468a-908d-ae951345e9df None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:55.842725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5ffe-7e42-468a-908d-ae951345e9df None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:55.877347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bf287-a69a-4fc3-a8b9-56637e7e7b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:55.877955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bf287-a69a-4fc3-a8b9-56637e7e7b38 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:55.878261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bf287-a69a-4fc3-a8b9-56637e7e7b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:55.878526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bf287-a69a-4fc3-a8b9-56637e7e7b38 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:55.879285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bf287-a69a-4fc3-a8b9-56637e7e7b38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:55.879597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bf287-a69a-4fc3-a8b9-56637e7e7b38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:55.881037 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bf287-a69a-4fc3-a8b9-56637e7e7b38 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:55.893645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 605/1231] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:12:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:55.899839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60b2e8-338e-4f0d-b482-6b617c000397 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:55.966394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:56.004060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60b2e8-338e-4f0d-b482-6b617c000397 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:56.037862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22791871-6516-4843-9d85-fe2dde1293af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:56.038613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22791871-6516-4843-9d85-fe2dde1293af None admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:56.038895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22791871-6516-4843-9d85-fe2dde1293af None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:56.039178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22791871-6516-4843-9d85-fe2dde1293af None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/244a8787df7641f2bac9b51213ddb322/roles/31125b687d5e4a29b891ade41748f51e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:56.054503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22791871-6516-4843-9d85-fe2dde1293af None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=244a8787df7641f2bac9b51213ddb322, role_id=31125b687d5e4a29b891ade41748f51e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:56.056059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22791871-6516-4843-9d85-fe2dde1293af None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:56.137556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 627/1232] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:12:55 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/244a8787df7641f2bac9b51213ddb322/roles/31125b687d5e4a29b891ade41748f51e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:12:58.134042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b550e-3f6b-4736-8b37-9b6e4fe0a315 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:58.134629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b550e-3f6b-4736-8b37-9b6e4fe0a315 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:58.134629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b550e-3f6b-4736-8b37-9b6e4fe0a315 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:58.135972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 606/1233] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:12:58.141476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9d750-24ba-498a-9081-92fb0203b6ce None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:58.141933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9d750-24ba-498a-9081-92fb0203b6ce None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:58.142169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9d750-24ba-498a-9081-92fb0203b6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:58.160350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-c3e9d750-24ba-498a-9081-92fb0203b6ce None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:12:58.161759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c3e9d750-24ba-498a-9081-92fb0203b6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:12:58.227118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 628/1234] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:12:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:12:58.252369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b550e-3f6b-4736-8b37-9b6e4fe0a315 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:58.278161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:58.355160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b550e-3f6b-4736-8b37-9b6e4fe0a315 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:58.390377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d65ee4-6cee-4bd5-b6ea-d64110a31c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:58.390941 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d65ee4-6cee-4bd5-b6ea-d64110a31c28 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:58.391224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d65ee4-6cee-4bd5-b6ea-d64110a31c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:58.391501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d65ee4-6cee-4bd5-b6ea-d64110a31c28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:58.404427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d65ee4-6cee-4bd5-b6ea-d64110a31c28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-18d65ee4-6cee-4bd5-b6ea-d64110a31c28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:12:58.409381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:58.410534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:12:58.411413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 607/1235] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 28 06:12:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:12:58.416620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9d750-24ba-498a-9081-92fb0203b6ce None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:58.467327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:58.519423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9d750-24ba-498a-9081-92fb0203b6ce None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:58.553482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056dad5-f63b-40df-99a3-545fa13adb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:58.554109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056dad5-f63b-40df-99a3-545fa13adb8b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:58.554400 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056dad5-f63b-40df-99a3-545fa13adb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:58.554684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056dad5-f63b-40df-99a3-545fa13adb8b None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:58.555403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056dad5-f63b-40df-99a3-545fa13adb8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:58.555718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056dad5-f63b-40df-99a3-545fa13adb8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:58.557098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056dad5-f63b-40df-99a3-545fa13adb8b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:58.564978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 629/1236] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 28 06:12:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:58.570976 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d65ee4-6cee-4bd5-b6ea-d64110a31c28 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:58.658615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:58.674514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d65ee4-6cee-4bd5-b6ea-d64110a31c28 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:58.708069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8d429-0195-43f8-87bf-134e47edcd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:58.708709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8d429-0195-43f8-87bf-134e47edcd8c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:58.708997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8d429-0195-43f8-87bf-134e47edcd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:58.709276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8d429-0195-43f8-87bf-134e47edcd8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:58.720400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8d429-0195-43f8-87bf-134e47edcd8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:58.723017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8d429-0195-43f8-87bf-134e47edcd8c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-95b8d429-0195-43f8-87bf-134e47edcd8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:12:58.735905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:58.736845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:58.737831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:12:58.737831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:12:58.737831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:58.737831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:12:58.737831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 608/1237] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:58.743407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056dad5-f63b-40df-99a3-545fa13adb8b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:58.845018 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056dad5-f63b-40df-99a3-545fa13adb8b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:58.853925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:58.878783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48285e8-dbbe-443a-b1d5-ce80b6451a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:58.879405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48285e8-dbbe-443a-b1d5-ce80b6451a79 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:58.879696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48285e8-dbbe-443a-b1d5-ce80b6451a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:58.879978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48285e8-dbbe-443a-b1d5-ce80b6451a79 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:58.880674 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48285e8-dbbe-443a-b1d5-ce80b6451a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:58.880986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48285e8-dbbe-443a-b1d5-ce80b6451a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:58.882403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48285e8-dbbe-443a-b1d5-ce80b6451a79 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:58.894836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 630/1238] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:58.900740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8d429-0195-43f8-87bf-134e47edcd8c None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:59.003952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8d429-0195-43f8-87bf-134e47edcd8c None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:59.038272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40b3af-1a8a-4ab0-8e62-74ec003292d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:59.038915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40b3af-1a8a-4ab0-8e62-74ec003292d2 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:59.039189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40b3af-1a8a-4ab0-8e62-74ec003292d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:59.039462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40b3af-1a8a-4ab0-8e62-74ec003292d2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:59.042154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:59.048112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40b3af-1a8a-4ab0-8e62-74ec003292d2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:59.049622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40b3af-1a8a-4ab0-8e62-74ec003292d2 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-5a40b3af-1a8a-4ab0-8e62-74ec003292d2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:12:59.060902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:59.062127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:12:59.063315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 609/1239] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:12:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:59.068187 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48285e8-dbbe-443a-b1d5-ce80b6451a79 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:59.169436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48285e8-dbbe-443a-b1d5-ce80b6451a79 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:59.203770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37895d18-3df0-4fb5-ba1b-268166dbcb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:59.204399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37895d18-3df0-4fb5-ba1b-268166dbcb56 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:59.204685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37895d18-3df0-4fb5-ba1b-268166dbcb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:59.204972 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37895d18-3df0-4fb5-ba1b-268166dbcb56 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:59.205813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37895d18-3df0-4fb5-ba1b-268166dbcb56 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:59.206127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37895d18-3df0-4fb5-ba1b-268166dbcb56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:59.207566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37895d18-3df0-4fb5-ba1b-268166dbcb56 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:59.225644 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 631/1240] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 28 06:12:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:59.232299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40b3af-1a8a-4ab0-8e62-74ec003292d2 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:59.338985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40b3af-1a8a-4ab0-8e62-74ec003292d2 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:59.373810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38894fe8-6587-4999-ab1a-82a6870202d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:59.374472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38894fe8-6587-4999-ab1a-82a6870202d1 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:59.374760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38894fe8-6587-4999-ab1a-82a6870202d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:59.375043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38894fe8-6587-4999-ab1a-82a6870202d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:59.444439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:59.453571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38894fe8-6587-4999-ab1a-82a6870202d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:59.455069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38894fe8-6587-4999-ab1a-82a6870202d1 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:59.573764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 610/1241] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:12:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:12:59.581444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37895d18-3df0-4fb5-ba1b-268166dbcb56 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:59.618498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:59.716398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37895d18-3df0-4fb5-ba1b-268166dbcb56 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:59.755170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3773486-c9e9-4c9d-8e69-93d88a4efcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:59.755828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3773486-c9e9-4c9d-8e69-93d88a4efcde None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:59.756118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3773486-c9e9-4c9d-8e69-93d88a4efcde None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:59.756399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3773486-c9e9-4c9d-8e69-93d88a4efcde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:59.767582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3773486-c9e9-4c9d-8e69-93d88a4efcde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:59.769075 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3773486-c9e9-4c9d-8e69-93d88a4efcde None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-c3773486-c9e9-4c9d-8e69-93d88a4efcde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:12:59.784394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:12:59.785854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:12:59.786788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:12:59.786788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:12:59.786788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:12:59.786788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:12:59.786788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 632/1242] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:12:59.792268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38894fe8-6587-4999-ab1a-82a6870202d1 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:12:59.894021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:12:59.901508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38894fe8-6587-4999-ab1a-82a6870202d1 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:12:59.936733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927958bf-05c6-44b4-bb09-745d436398aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:12:59.937455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927958bf-05c6-44b4-bb09-745d436398aa None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:12:59.937745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927958bf-05c6-44b4-bb09-745d436398aa None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:12:59.938032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927958bf-05c6-44b4-bb09-745d436398aa None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:12:59.938735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927958bf-05c6-44b4-bb09-745d436398aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:12:59.939046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927958bf-05c6-44b4-bb09-745d436398aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:12:59.940448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927958bf-05c6-44b4-bb09-745d436398aa None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:12:59.957221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 611/1243] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:12:59.959093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3773486-c9e9-4c9d-8e69-93d88a4efcde None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:00.054741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:00.085563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3773486-c9e9-4c9d-8e69-93d88a4efcde None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:00.130116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4c64d-4052-4032-8468-bc77dc63b0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:00.130989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4c64d-4052-4032-8468-bc77dc63b0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:00.131489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4c64d-4052-4032-8468-bc77dc63b0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:00.132066 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4c64d-4052-4032-8468-bc77dc63b0f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:00.140342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4c64d-4052-4032-8468-bc77dc63b0f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:00.141902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4c64d-4052-4032-8468-bc77dc63b0f8 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-73f4c64d-4052-4032-8468-bc77dc63b0f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:13:00.159081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:00.161594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:00.167625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:13:00.167625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:13:00.167625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:13:00.167625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:13:00.167625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:13:00.167625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:13:00.167625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:13:00.167625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:00.167625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 633/1244] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:12:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:00.168393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927958bf-05c6-44b4-bb09-745d436398aa None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:00.270239 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927958bf-05c6-44b4-bb09-745d436398aa None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:00.303843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fdcc9-ef04-4eae-93d9-86b9c6b8eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:00.304467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fdcc9-ef04-4eae-93d9-86b9c6b8eff6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:00.305880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fdcc9-ef04-4eae-93d9-86b9c6b8eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:00.305880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fdcc9-ef04-4eae-93d9-86b9c6b8eff6 None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:00.305880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fdcc9-ef04-4eae-93d9-86b9c6b8eff6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:00.306230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fdcc9-ef04-4eae-93d9-86b9c6b8eff6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:00.307538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fdcc9-ef04-4eae-93d9-86b9c6b8eff6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:00.319339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 612/1245] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:13:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:00.326055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4c64d-4052-4032-8468-bc77dc63b0f8 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:00.431930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4c64d-4052-4032-8468-bc77dc63b0f8 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:00.465781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5b972-a7dc-4e0a-ba46-7525007728cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:00.466626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5b972-a7dc-4e0a-ba46-7525007728cd None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:00.466909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5b972-a7dc-4e0a-ba46-7525007728cd None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:00.467183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5b972-a7dc-4e0a-ba46-7525007728cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:00.467970 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5b972-a7dc-4e0a-ba46-7525007728cd None admin] RBAC: Adding query filter params (role.id=31125b687d5e4a29b891ade41748f51e, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=244a8787df7641f2bac9b51213ddb322) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:00.468280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5b972-a7dc-4e0a-ba46-7525007728cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:00.469740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5b972-a7dc-4e0a-ba46-7525007728cd None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:00.475676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 634/1246] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:13:00 2022] GET /identity/v3/role_assignments?user.id=244a8787df7641f2bac9b51213ddb322&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=31125b687d5e4a29b891ade41748f51e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:02.623352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570eec4-f054-4243-8f55-cc7744cb0d3f None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:02.623862 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570eec4-f054-4243-8f55-cc7744cb0d3f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:02.623862 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570eec4-f054-4243-8f55-cc7744cb0d3f None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:02.625246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 613/1247] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:13:02.630742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252dfb69-f953-4c38-a6bb-1070bd6c1692 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:02.631185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252dfb69-f953-4c38-a6bb-1070bd6c1692 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:02.631417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252dfb69-f953-4c38-a6bb-1070bd6c1692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:02.637948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:02.650232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-252dfb69-f953-4c38-a6bb-1070bd6c1692 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:13:02.651429 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-252dfb69-f953-4c38-a6bb-1070bd6c1692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:13:02.716458 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 635/1248] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:13:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:13:02.740202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570eec4-f054-4243-8f55-cc7744cb0d3f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:02.759399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:02.843491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570eec4-f054-4243-8f55-cc7744cb0d3f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:02.878620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d59a0-e6e0-4771-b22a-1ce93edb2510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:02.879190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d59a0-e6e0-4771-b22a-1ce93edb2510 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:02.879478 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d59a0-e6e0-4771-b22a-1ce93edb2510 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:02.879757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d59a0-e6e0-4771-b22a-1ce93edb2510 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:02.890623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d59a0-e6e0-4771-b22a-1ce93edb2510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-3d2d59a0-e6e0-4771-b22a-1ce93edb2510 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:13:02.895516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:02.896616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:13:02.897569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 614/1249] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:13:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:02.903037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252dfb69-f953-4c38-a6bb-1070bd6c1692 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:03.002992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252dfb69-f953-4c38-a6bb-1070bd6c1692 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:03.032295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:03.037074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b83676-91b7-42d8-be4a-269e0d7fbb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:03.037735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b83676-91b7-42d8-be4a-269e0d7fbb8a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:03.038028 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b83676-91b7-42d8-be4a-269e0d7fbb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:03.038311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b83676-91b7-42d8-be4a-269e0d7fbb8a None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:03.039034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b83676-91b7-42d8-be4a-269e0d7fbb8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:03.039346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b83676-91b7-42d8-be4a-269e0d7fbb8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:03.040731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b83676-91b7-42d8-be4a-269e0d7fbb8a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:03.048831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 636/1250] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:13:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:03.054504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d59a0-e6e0-4771-b22a-1ce93edb2510 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:03.140955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:03.165767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d59a0-e6e0-4771-b22a-1ce93edb2510 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:03.202844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010003f-ffe8-418d-911d-9ca0104f1061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:03.203502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010003f-ffe8-418d-911d-9ca0104f1061 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:03.203791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010003f-ffe8-418d-911d-9ca0104f1061 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:03.204071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010003f-ffe8-418d-911d-9ca0104f1061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:03.214777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010003f-ffe8-418d-911d-9ca0104f1061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:03.216385 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010003f-ffe8-418d-911d-9ca0104f1061 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-4010003f-ffe8-418d-911d-9ca0104f1061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:13:03.233751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:03.239642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:03.240792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:13:03.240792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:13:03.240792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:03.240792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:13:03.240792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 615/1251] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:03.241926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b83676-91b7-42d8-be4a-269e0d7fbb8a None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:03.345296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b83676-91b7-42d8-be4a-269e0d7fbb8a None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:03.386625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b6947-ab42-42f3-bf08-fa8d4f21da06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:03.387252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b6947-ab42-42f3-bf08-fa8d4f21da06 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:03.387541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b6947-ab42-42f3-bf08-fa8d4f21da06 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:03.387822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b6947-ab42-42f3-bf08-fa8d4f21da06 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:03.388521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b6947-ab42-42f3-bf08-fa8d4f21da06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:03.388830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b6947-ab42-42f3-bf08-fa8d4f21da06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:03.390245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b6947-ab42-42f3-bf08-fa8d4f21da06 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:03.402419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 637/1252] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:13:03.408000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010003f-ffe8-418d-911d-9ca0104f1061 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:03.509976 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010003f-ffe8-418d-911d-9ca0104f1061 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:03.537206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:03.545514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752fab6-5b7e-4ed0-a4cf-b9bfbd4e0847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:03.546177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752fab6-5b7e-4ed0-a4cf-b9bfbd4e0847 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:03.546506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752fab6-5b7e-4ed0-a4cf-b9bfbd4e0847 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:03.546725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752fab6-5b7e-4ed0-a4cf-b9bfbd4e0847 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:03.554818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752fab6-5b7e-4ed0-a4cf-b9bfbd4e0847 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:03.556414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752fab6-5b7e-4ed0-a4cf-b9bfbd4e0847 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-1752fab6-5b7e-4ed0-a4cf-b9bfbd4e0847 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:13:03.568712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:03.570031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:13:03.571173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 616/1253] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:13:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:03.576517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b6947-ab42-42f3-bf08-fa8d4f21da06 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:03.642063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:03.687760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b6947-ab42-42f3-bf08-fa8d4f21da06 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:03.722622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ad0e4-4691-4c0f-91b7-be9ecc105b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:03.723279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ad0e4-4691-4c0f-91b7-be9ecc105b18 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:03.723571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ad0e4-4691-4c0f-91b7-be9ecc105b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:03.723853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ad0e4-4691-4c0f-91b7-be9ecc105b18 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:03.724650 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ad0e4-4691-4c0f-91b7-be9ecc105b18 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:03.724970 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ad0e4-4691-4c0f-91b7-be9ecc105b18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:03.726441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ad0e4-4691-4c0f-91b7-be9ecc105b18 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:03.743841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 638/1254] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 28 06:13:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:03.750183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752fab6-5b7e-4ed0-a4cf-b9bfbd4e0847 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:03.850962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752fab6-5b7e-4ed0-a4cf-b9bfbd4e0847 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:03.886391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3e5e2-5fec-42b4-b5f3-6eedd4395633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:03.887169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3e5e2-5fec-42b4-b5f3-6eedd4395633 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:03.887461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3e5e2-5fec-42b4-b5f3-6eedd4395633 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:03.887737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3e5e2-5fec-42b4-b5f3-6eedd4395633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:03.946567 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:03.959992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3e5e2-5fec-42b4-b5f3-6eedd4395633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:03.962196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3e5e2-5fec-42b4-b5f3-6eedd4395633 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:04.061254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 617/1255] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:13:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:13:04.067142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ad0e4-4691-4c0f-91b7-be9ecc105b18 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:04.167489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ad0e4-4691-4c0f-91b7-be9ecc105b18 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:04.180775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:04.202136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23c379-dd8a-42a3-ba7d-9719143c1d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:04.202427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23c379-dd8a-42a3-ba7d-9719143c1d9e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:04.202715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23c379-dd8a-42a3-ba7d-9719143c1d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:04.202994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23c379-dd8a-42a3-ba7d-9719143c1d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:04.212774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23c379-dd8a-42a3-ba7d-9719143c1d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:04.214256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23c379-dd8a-42a3-ba7d-9719143c1d9e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-1c23c379-dd8a-42a3-ba7d-9719143c1d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:13:04.229278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:04.231959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:04.233856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:13:04.233856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:13:04.233856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:04.233856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:04.233856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 639/1256] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:04.237022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3e5e2-5fec-42b4-b5f3-6eedd4395633 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:04.325004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:04.336836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3e5e2-5fec-42b4-b5f3-6eedd4395633 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:04.372710 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e11f3-32bd-4806-a95b-37280b5d7bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:04.373353 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e11f3-32bd-4806-a95b-37280b5d7bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:04.373644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e11f3-32bd-4806-a95b-37280b5d7bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:04.373922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e11f3-32bd-4806-a95b-37280b5d7bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:04.374645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e11f3-32bd-4806-a95b-37280b5d7bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:04.374953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e11f3-32bd-4806-a95b-37280b5d7bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:04.376633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e11f3-32bd-4806-a95b-37280b5d7bd3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:04.388970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 618/1257] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:04.394586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23c379-dd8a-42a3-ba7d-9719143c1d9e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:04.495938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23c379-dd8a-42a3-ba7d-9719143c1d9e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:04.529179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33f2b8-9f21-45cc-98d9-1de90b551d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:04.530377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33f2b8-9f21-45cc-98d9-1de90b551d62 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:04.530663 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33f2b8-9f21-45cc-98d9-1de90b551d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:04.530949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33f2b8-9f21-45cc-98d9-1de90b551d62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:04.537639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33f2b8-9f21-45cc-98d9-1de90b551d62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:04.539095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33f2b8-9f21-45cc-98d9-1de90b551d62 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-6e33f2b8-9f21-45cc-98d9-1de90b551d62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:13:04.550783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:04.551857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:04.557841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:13:04.557841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:13:04.557841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:13:04.557841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:13:04.557841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:13:04.557841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:13:04.557841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:13:04.557841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:04.557841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 640/1258] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:13:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:04.558466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e11f3-32bd-4806-a95b-37280b5d7bd3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:04.659221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e11f3-32bd-4806-a95b-37280b5d7bd3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:04.693439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7c495-d730-4dc5-a1ee-9683d07fea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:04.694060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c495-d730-4dc5-a1ee-9683d07fea9e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:04.694342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c495-d730-4dc5-a1ee-9683d07fea9e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:04.694618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c495-d730-4dc5-a1ee-9683d07fea9e None admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:04.695364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7c495-d730-4dc5-a1ee-9683d07fea9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:04.695666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7c495-d730-4dc5-a1ee-9683d07fea9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:04.697098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7c495-d730-4dc5-a1ee-9683d07fea9e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:04.708971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 619/1259] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:13:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:04.715290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33f2b8-9f21-45cc-98d9-1de90b551d62 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:04.742823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:04.816935 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33f2b8-9f21-45cc-98d9-1de90b551d62 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:04.850727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78264c83-b340-419c-8f49-4256bced9b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:04.851350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264c83-b340-419c-8f49-4256bced9b1a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:04.851640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264c83-b340-419c-8f49-4256bced9b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:04.851923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264c83-b340-419c-8f49-4256bced9b1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:04.852719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78264c83-b340-419c-8f49-4256bced9b1a None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=244a8787df7641f2bac9b51213ddb322) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:04.853035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78264c83-b340-419c-8f49-4256bced9b1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:04.854562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78264c83-b340-419c-8f49-4256bced9b1a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:04.860698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 641/1260] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:13:04 2022] GET /identity/v3/role_assignments?user.id=244a8787df7641f2bac9b51213ddb322&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:06.857967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac45709-d5db-4c73-a756-6a3f03ce21eb None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:06.858676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac45709-d5db-4c73-a756-6a3f03ce21eb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:06.858676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac45709-d5db-4c73-a756-6a3f03ce21eb None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:06.859904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 620/1261] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:13:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:13:06.865019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380fb5e-5877-4851-9de1-2e0c49fe37aa None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:06.865451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380fb5e-5877-4851-9de1-2e0c49fe37aa None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:06.865707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380fb5e-5877-4851-9de1-2e0c49fe37aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:06.884519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-d380fb5e-5877-4851-9de1-2e0c49fe37aa None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:13:06.887627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d380fb5e-5877-4851-9de1-2e0c49fe37aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:13:06.953748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 642/1262] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:13:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:13:06.976134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac45709-d5db-4c73-a756-6a3f03ce21eb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:06.987443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:07.094582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac45709-d5db-4c73-a756-6a3f03ce21eb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:07.129103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e9b95-21bc-4f1e-8baf-d43b91628669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:07.129696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e9b95-21bc-4f1e-8baf-d43b91628669 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:07.129983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e9b95-21bc-4f1e-8baf-d43b91628669 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:07.130261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e9b95-21bc-4f1e-8baf-d43b91628669 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:07.140913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e9b95-21bc-4f1e-8baf-d43b91628669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-976e9b95-21bc-4f1e-8baf-d43b91628669 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:13:07.145817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:07.147038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:13:07.148075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 621/1263] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:13:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:07.153110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380fb5e-5877-4851-9de1-2e0c49fe37aa None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:07.172862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:07.256211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380fb5e-5877-4851-9de1-2e0c49fe37aa None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:07.290676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3248f-37f6-4d28-951f-71581c409672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:07.291301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3248f-37f6-4d28-951f-71581c409672 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:07.291589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3248f-37f6-4d28-951f-71581c409672 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:07.291870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3248f-37f6-4d28-951f-71581c409672 None admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:07.292597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3248f-37f6-4d28-951f-71581c409672 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:07.292901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3248f-37f6-4d28-951f-71581c409672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:07.294308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3248f-37f6-4d28-951f-71581c409672 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:07.303241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 643/1264] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:13:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:07.309038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e9b95-21bc-4f1e-8baf-d43b91628669 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:07.380730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:07.415641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e9b95-21bc-4f1e-8baf-d43b91628669 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:07.450100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c27162-8c57-4008-888c-4f1c98a12c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:07.450735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c27162-8c57-4008-888c-4f1c98a12c44 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:07.451029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c27162-8c57-4008-888c-4f1c98a12c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:07.451381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c27162-8c57-4008-888c-4f1c98a12c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:07.462180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c27162-8c57-4008-888c-4f1c98a12c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:07.463641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c27162-8c57-4008-888c-4f1c98a12c44 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-e3c27162-8c57-4008-888c-4f1c98a12c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:13:07.478918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:07.483433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:07.484736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:13:07.484736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:13:07.484736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:07.484736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:13:07.484736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 622/1265] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:07.486629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3248f-37f6-4d28-951f-71581c409672 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:07.574390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:07.594571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3248f-37f6-4d28-951f-71581c409672 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:07.633644 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40775f3b-797b-4329-8264-6de7da1a9b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:07.634182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40775f3b-797b-4329-8264-6de7da1a9b2b None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:07.634473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40775f3b-797b-4329-8264-6de7da1a9b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:07.634756 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40775f3b-797b-4329-8264-6de7da1a9b2b None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:07.635459 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40775f3b-797b-4329-8264-6de7da1a9b2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:07.635772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40775f3b-797b-4329-8264-6de7da1a9b2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:07.637153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40775f3b-797b-4329-8264-6de7da1a9b2b None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:07.649182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 644/1266] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:07.655080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c27162-8c57-4008-888c-4f1c98a12c44 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:07.756193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c27162-8c57-4008-888c-4f1c98a12c44 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:07.775471 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:07.789034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fef23a-69f9-4182-b30c-a5cdfda92bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:07.789752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fef23a-69f9-4182-b30c-a5cdfda92bfb None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:07.790042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fef23a-69f9-4182-b30c-a5cdfda92bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:07.790326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fef23a-69f9-4182-b30c-a5cdfda92bfb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:07.799230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fef23a-69f9-4182-b30c-a5cdfda92bfb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:07.800689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fef23a-69f9-4182-b30c-a5cdfda92bfb None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-09fef23a-69f9-4182-b30c-a5cdfda92bfb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:13:07.812532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:07.813716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:13:07.814832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 623/1267] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:13:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:07.819682 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40775f3b-797b-4329-8264-6de7da1a9b2b None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:07.926319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40775f3b-797b-4329-8264-6de7da1a9b2b None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:07.960643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4846383-bf8d-4ff6-9f8a-011de8b5da59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:07.961263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4846383-bf8d-4ff6-9f8a-011de8b5da59 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:07.961612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4846383-bf8d-4ff6-9f8a-011de8b5da59 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:07.961924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4846383-bf8d-4ff6-9f8a-011de8b5da59 None admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:07.962830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4846383-bf8d-4ff6-9f8a-011de8b5da59 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:07.963148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4846383-bf8d-4ff6-9f8a-011de8b5da59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:07.964576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4846383-bf8d-4ff6-9f8a-011de8b5da59 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:07.973973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:07.983097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 645/1268] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 28 06:13:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:07.988765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fef23a-69f9-4182-b30c-a5cdfda92bfb None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:08.090180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fef23a-69f9-4182-b30c-a5cdfda92bfb None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:08.124466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2aaf7-144d-4974-8713-c75ba8cba806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:08.125107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2aaf7-144d-4974-8713-c75ba8cba806 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:08.125423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2aaf7-144d-4974-8713-c75ba8cba806 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:08.125707 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2aaf7-144d-4974-8713-c75ba8cba806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:08.136561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2aaf7-144d-4974-8713-c75ba8cba806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:08.138051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2aaf7-144d-4974-8713-c75ba8cba806 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-6ed2aaf7-144d-4974-8713-c75ba8cba806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:13:08.152190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:08.153323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:08.154932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:13:08.154932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:13:08.154932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:08.154932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:13:08.154932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 624/1269] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:08.159757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4846383-bf8d-4ff6-9f8a-011de8b5da59 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:08.259928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4846383-bf8d-4ff6-9f8a-011de8b5da59 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:08.294001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbcd55-e878-410d-a28c-c96113ac32f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:08.294613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbcd55-e878-410d-a28c-c96113ac32f0 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:08.295676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbcd55-e878-410d-a28c-c96113ac32f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:08.296061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbcd55-e878-410d-a28c-c96113ac32f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:08.296752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbcd55-e878-410d-a28c-c96113ac32f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:08.297056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbcd55-e878-410d-a28c-c96113ac32f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:08.298453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbcd55-e878-410d-a28c-c96113ac32f0 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:08.309217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 646/1270] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:08.315073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2aaf7-144d-4974-8713-c75ba8cba806 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:08.348098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:08.426378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2aaf7-144d-4974-8713-c75ba8cba806 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:08.461442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb9ae-d1a9-46e1-90c7-3f26a94fca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:08.462080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb9ae-d1a9-46e1-90c7-3f26a94fca00 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:08.462369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb9ae-d1a9-46e1-90c7-3f26a94fca00 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:08.462633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb9ae-d1a9-46e1-90c7-3f26a94fca00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:08.470050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fb9ae-d1a9-46e1-90c7-3f26a94fca00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:08.471500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fb9ae-d1a9-46e1-90c7-3f26a94fca00 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b80fb9ae-d1a9-46e1-90c7-3f26a94fca00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:13:08.482899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:08.484074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:08.485019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:13:08.485019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:13:08.485019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:13:08.485019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:13:08.485019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:13:08.485019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:13:08.485019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:13:08.485019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:13:08.485019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 625/1271] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:13:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:08.491353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbcd55-e878-410d-a28c-c96113ac32f0 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:08.548688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:08.590422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbcd55-e878-410d-a28c-c96113ac32f0 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:08.624468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a359c-6da6-4dca-8a18-c76b3342bb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:08.625094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a359c-6da6-4dca-8a18-c76b3342bb56 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:08.625413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a359c-6da6-4dca-8a18-c76b3342bb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:08.625700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a359c-6da6-4dca-8a18-c76b3342bb56 None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:08.626782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a359c-6da6-4dca-8a18-c76b3342bb56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:08.627101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a359c-6da6-4dca-8a18-c76b3342bb56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:08.628536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a359c-6da6-4dca-8a18-c76b3342bb56 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:08.642069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 647/1272] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:13:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:08.648332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb9ae-d1a9-46e1-90c7-3f26a94fca00 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:08.743501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:08.752827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb9ae-d1a9-46e1-90c7-3f26a94fca00 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:08.786531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d76f4-9fc4-4a8f-bbf8-c4776d9ca280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:08.787199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d76f4-9fc4-4a8f-bbf8-c4776d9ca280 None admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:08.787486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d76f4-9fc4-4a8f-bbf8-c4776d9ca280 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:08.787813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d76f4-9fc4-4a8f-bbf8-c4776d9ca280 None admin] PATH_INFO: `/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/244a8787df7641f2bac9b51213ddb322/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:08.789731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d76f4-9fc4-4a8f-bbf8-c4776d9ca280 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0603e8736d4dfb846b4005972563f4, user_id=244a8787df7641f2bac9b51213ddb322, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:08.791180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d76f4-9fc4-4a8f-bbf8-c4776d9ca280 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:08.896804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 626/1273] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 28 06:13:08 2022] PUT /identity/v3/projects/4a0603e8736d4dfb846b4005972563f4/users/244a8787df7641f2bac9b51213ddb322/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:13:10.896717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5d49a-da4c-4ba2-bc2e-f8681e6a8705 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:10.897250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5d49a-da4c-4ba2-bc2e-f8681e6a8705 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:10.897405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5d49a-da4c-4ba2-bc2e-f8681e6a8705 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:10.898676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 648/1274] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:13:10.904105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aacd6-35c4-4924-9c57-4dff176054aa None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:10.904514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aacd6-35c4-4924-9c57-4dff176054aa None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:10.904838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aacd6-35c4-4924-9c57-4dff176054aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:10.923889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7b8aacd6-35c4-4924-9c57-4dff176054aa None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:13:10.928205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7b8aacd6-35c4-4924-9c57-4dff176054aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:13:10.991630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 627/1275] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:13:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:13:11.014759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5d49a-da4c-4ba2-bc2e-f8681e6a8705 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:11.116224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5d49a-da4c-4ba2-bc2e-f8681e6a8705 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:11.133772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:11.150226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a737478-70e7-4701-8165-2b4fbf074a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:11.150799 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a737478-70e7-4701-8165-2b4fbf074a02 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:11.151088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a737478-70e7-4701-8165-2b4fbf074a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:11.151371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a737478-70e7-4701-8165-2b4fbf074a02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:11.162042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a737478-70e7-4701-8165-2b4fbf074a02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-5a737478-70e7-4701-8165-2b4fbf074a02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:13:11.166957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:11.167983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:11.168827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 649/1276] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 28 06:13:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:11.174950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8aacd6-35c4-4924-9c57-4dff176054aa None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:11.257631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:11.275830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8aacd6-35c4-4924-9c57-4dff176054aa None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:11.309064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54285de-0fc1-45c2-828c-16c39c4be927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:11.309711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54285de-0fc1-45c2-828c-16c39c4be927 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:11.309996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54285de-0fc1-45c2-828c-16c39c4be927 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:11.310808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54285de-0fc1-45c2-828c-16c39c4be927 None admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:11.311722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54285de-0fc1-45c2-828c-16c39c4be927 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:11.312030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54285de-0fc1-45c2-828c-16c39c4be927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:11.313440 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54285de-0fc1-45c2-828c-16c39c4be927 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:11.321686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 628/1277] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 28 06:13:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:11.328361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a737478-70e7-4701-8165-2b4fbf074a02 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:11.431193 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a737478-70e7-4701-8165-2b4fbf074a02 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:11.471829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70c03e-5cd9-48ce-9c6f-9ede2a532d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:11.472482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70c03e-5cd9-48ce-9c6f-9ede2a532d1f None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:11.472774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70c03e-5cd9-48ce-9c6f-9ede2a532d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:11.473054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70c03e-5cd9-48ce-9c6f-9ede2a532d1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:11.483906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70c03e-5cd9-48ce-9c6f-9ede2a532d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:11.485392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70c03e-5cd9-48ce-9c6f-9ede2a532d1f None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-8b70c03e-5cd9-48ce-9c6f-9ede2a532d1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:13:11.501176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:11.502308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:11.503450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:13:11.503450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:13:11.503450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:11.503450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:11.503450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 650/1278] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:11.508396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54285de-0fc1-45c2-828c-16c39c4be927 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:11.612393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54285de-0fc1-45c2-828c-16c39c4be927 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:11.648321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaae73-7124-4eed-9dc4-f8cf20ad75e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:11.648952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaae73-7124-4eed-9dc4-f8cf20ad75e3 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:11.649240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaae73-7124-4eed-9dc4-f8cf20ad75e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:11.649550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaae73-7124-4eed-9dc4-f8cf20ad75e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:11.650258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aaae73-7124-4eed-9dc4-f8cf20ad75e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:11.650569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aaae73-7124-4eed-9dc4-f8cf20ad75e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:11.651957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aaae73-7124-4eed-9dc4-f8cf20ad75e3 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:11.664043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 629/1279] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:11.670499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70c03e-5cd9-48ce-9c6f-9ede2a532d1f None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:11.703097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:11.775823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70c03e-5cd9-48ce-9c6f-9ede2a532d1f None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:11.809216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c28b08-3889-452f-99d8-e2f9f07a5fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:11.809922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c28b08-3889-452f-99d8-e2f9f07a5fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:11.810686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c28b08-3889-452f-99d8-e2f9f07a5fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:11.810987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c28b08-3889-452f-99d8-e2f9f07a5fe5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:11.817974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c28b08-3889-452f-99d8-e2f9f07a5fe5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:11.819371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c28b08-3889-452f-99d8-e2f9f07a5fe5 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-73c28b08-3889-452f-99d8-e2f9f07a5fe5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:13:11.831752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:11.832899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:11.834273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 651/1280] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:13:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:11.838795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaae73-7124-4eed-9dc4-f8cf20ad75e3 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:11.843379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:11.940290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaae73-7124-4eed-9dc4-f8cf20ad75e3 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:11.974743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4047963-9768-4edb-8cb9-54aa60c3f8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:11.975293 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4047963-9768-4edb-8cb9-54aa60c3f8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:11.975694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4047963-9768-4edb-8cb9-54aa60c3f8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:11.975975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4047963-9768-4edb-8cb9-54aa60c3f8d6 None admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:11.976771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4047963-9768-4edb-8cb9-54aa60c3f8d6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:11.977080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4047963-9768-4edb-8cb9-54aa60c3f8d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:11.978550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4047963-9768-4edb-8cb9-54aa60c3f8d6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:11.995261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 630/1281] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 28 06:13:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:12.002004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c28b08-3889-452f-99d8-e2f9f07a5fe5 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:12.099189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:12.109211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c28b08-3889-452f-99d8-e2f9f07a5fe5 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:12.142823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897b9ec-6465-4464-a8d8-30bac265c3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:12.143449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897b9ec-6465-4464-a8d8-30bac265c3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:12.143741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897b9ec-6465-4464-a8d8-30bac265c3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:12.144017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897b9ec-6465-4464-a8d8-30bac265c3a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:12.213973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897b9ec-6465-4464-a8d8-30bac265c3a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:12.215478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897b9ec-6465-4464-a8d8-30bac265c3a4 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:12.304354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:12.315024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 652/1282] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 28 06:13:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:13:12.320776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4047963-9768-4edb-8cb9-54aa60c3f8d6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:12.384611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:12.430019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4047963-9768-4edb-8cb9-54aa60c3f8d6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:12.465075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012221f-0c88-48ca-9d01-844b775c6290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:12.465745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012221f-0c88-48ca-9d01-844b775c6290 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:12.466035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012221f-0c88-48ca-9d01-844b775c6290 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:12.466924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012221f-0c88-48ca-9d01-844b775c6290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:12.477556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012221f-0c88-48ca-9d01-844b775c6290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:12.479052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012221f-0c88-48ca-9d01-844b775c6290 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-5012221f-0c88-48ca-9d01-844b775c6290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:13:12.494275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:12.495417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:12.496396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:13:12.496396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:13:12.496396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:13:12.496396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:13:12.496396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 631/1283] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:12.502256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897b9ec-6465-4464-a8d8-30bac265c3a4 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:12.606686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897b9ec-6465-4464-a8d8-30bac265c3a4 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:12.640528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e4574-815e-4b96-a7ac-512fc1dcf590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:12.641139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e4574-815e-4b96-a7ac-512fc1dcf590 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:12.641450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e4574-815e-4b96-a7ac-512fc1dcf590 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:12.641727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e4574-815e-4b96-a7ac-512fc1dcf590 None admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:12.642415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e4574-815e-4b96-a7ac-512fc1dcf590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:12.642726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e4574-815e-4b96-a7ac-512fc1dcf590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:12.644124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e4574-815e-4b96-a7ac-512fc1dcf590 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:12.655364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 653/1284] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 28 06:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:12.661077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012221f-0c88-48ca-9d01-844b775c6290 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:12.771158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012221f-0c88-48ca-9d01-844b775c6290 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:12.784997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:12.804268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b07000-2f62-48ce-93eb-d2202668583e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:12.804911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b07000-2f62-48ce-93eb-d2202668583e None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:12.805202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b07000-2f62-48ce-93eb-d2202668583e None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:12.805526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b07000-2f62-48ce-93eb-d2202668583e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:12.812994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b07000-2f62-48ce-93eb-d2202668583e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:12.814492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b07000-2f62-48ce-93eb-d2202668583e None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application [None req-b1b07000-2f62-48ce-93eb-d2202668583e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:13:12.827435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:12.830648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:12.831728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:13:12.831728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:13:12.831728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:13:12.831728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:13:12.831728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:13:12.831728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:13:12.831728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:13:12.831728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: ERROR keystone.server.flask.application Jul 28 06:13:12.831728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 632/1285] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 28 06:13:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:12.834753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e4574-815e-4b96-a7ac-512fc1dcf590 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:12.886535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:12.936496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e4574-815e-4b96-a7ac-512fc1dcf590 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:12.970569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c72c30-58da-4d21-ae07-160aabeb6d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:12.971185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c72c30-58da-4d21-ae07-160aabeb6d5a None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:12.971468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c72c30-58da-4d21-ae07-160aabeb6d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:12.971743 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c72c30-58da-4d21-ae07-160aabeb6d5a None admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:12.972498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c72c30-58da-4d21-ae07-160aabeb6d5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:12.972798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c72c30-58da-4d21-ae07-160aabeb6d5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:12.974311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c72c30-58da-4d21-ae07-160aabeb6d5a None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:12.986874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 654/1286] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 28 06:13:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:13:12.993153 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b07000-2f62-48ce-93eb-d2202668583e None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:13.095929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b07000-2f62-48ce-93eb-d2202668583e None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:13.130465 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dff512-df3b-4cef-af8c-db7d6981ea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:13.131090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dff512-df3b-4cef-af8c-db7d6981ea74 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:13.131373 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dff512-df3b-4cef-af8c-db7d6981ea74 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:13.131653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dff512-df3b-4cef-af8c-db7d6981ea74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:13.132445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dff512-df3b-4cef-af8c-db7d6981ea74 None admin] RBAC: Adding query filter params (role.id=728f3655cf7a47fc9e88157b0dd7dca4, scope.project.id=4a0603e8736d4dfb846b4005972563f4, user.id=244a8787df7641f2bac9b51213ddb322) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:13.132758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dff512-df3b-4cef-af8c-db7d6981ea74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:13.134270 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dff512-df3b-4cef-af8c-db7d6981ea74 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:13.141025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 633/1287] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 28 06:13:12 2022] GET /identity/v3/role_assignments?user.id=244a8787df7641f2bac9b51213ddb322&scope.project.id=4a0603e8736d4dfb846b4005972563f4&role.id=728f3655cf7a47fc9e88157b0dd7dca4 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:15.190300 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755a105-003d-479a-8b37-fe1eccce8ecf None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:15.190861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755a105-003d-479a-8b37-fe1eccce8ecf None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:15.190861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755a105-003d-479a-8b37-fe1eccce8ecf None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:15.192285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 655/1288] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:13:15.197316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a8d84-1d96-4faf-9b5c-c9b268cdf2a8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:15.197749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a8d84-1d96-4faf-9b5c-c9b268cdf2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:15.197883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a8d84-1d96-4faf-9b5c-c9b268cdf2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:15.216347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1b3a8d84-1d96-4faf-9b5c-c9b268cdf2a8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:13:15.217600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1b3a8d84-1d96-4faf-9b5c-c9b268cdf2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:13:15.246443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:15.283769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 634/1289] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:13:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:13:15.305349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755a105-003d-479a-8b37-fe1eccce8ecf None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:15.423984 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755a105-003d-479a-8b37-fe1eccce8ecf None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:15.434836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:15.459096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618baf83-b9bb-4c96-9833-bab6c7631f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:15.459665 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618baf83-b9bb-4c96-9833-bab6c7631f64 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:15.460033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618baf83-b9bb-4c96-9833-bab6c7631f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:15.460317 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618baf83-b9bb-4c96-9833-bab6c7631f64 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:15.460945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618baf83-b9bb-4c96-9833-bab6c7631f64 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:15.462374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618baf83-b9bb-4c96-9833-bab6c7631f64 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-618baf83-b9bb-4c96-9833-bab6c7631f64 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:15.471770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:15.473013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:15.473932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 656/1290] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 28 06:13:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:15.479238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a8d84-1d96-4faf-9b5c-c9b268cdf2a8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:15.580313 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a8d84-1d96-4faf-9b5c-c9b268cdf2a8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:15.616871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2d044-9050-4f04-b661-ebb826b23ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:15.617576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2d044-9050-4f04-b661-ebb826b23ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:15.618095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2d044-9050-4f04-b661-ebb826b23ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:15.618594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2d044-9050-4f04-b661-ebb826b23ad6 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:15.619950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2d044-9050-4f04-b661-ebb826b23ad6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:15.620333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2d044-9050-4f04-b661-ebb826b23ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:15.621768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2d044-9050-4f04-b661-ebb826b23ad6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:15.627885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 635/1291] 10.222.0.69 () {56 vars in 1116 bytes} [Thu Jul 28 06:13:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:15.633780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618baf83-b9bb-4c96-9833-bab6c7631f64 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:15.734981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618baf83-b9bb-4c96-9833-bab6c7631f64 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:15.770269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910a034-09dd-40fa-b698-52857f43b9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:15.774477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910a034-09dd-40fa-b698-52857f43b9ef None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:15.774477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910a034-09dd-40fa-b698-52857f43b9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:15.774477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910a034-09dd-40fa-b698-52857f43b9ef None admin] PATH_INFO: `/v3/services` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:15.774477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910a034-09dd-40fa-b698-52857f43b9ef None admin] RBAC: Adding query filter params (type=placement) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:15.774477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910a034-09dd-40fa-b698-52857f43b9ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:15.774477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910a034-09dd-40fa-b698-52857f43b9ef None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:15.778326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 657/1292] 10.222.0.69 () {56 vars in 1116 bytes} [Thu Jul 28 06:13:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:17.818763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646d77-9e8f-4253-8a0c-bb6905f5d931 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:17.819317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646d77-9e8f-4253-8a0c-bb6905f5d931 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:17.819414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646d77-9e8f-4253-8a0c-bb6905f5d931 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:17.820703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 636/1293] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:13:17.826656 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963a4c-ad77-46b3-a290-3935d7c12900 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:17.827076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963a4c-ad77-46b3-a290-3935d7c12900 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:17.827208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963a4c-ad77-46b3-a290-3935d7c12900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:17.845252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-af963a4c-ad77-46b3-a290-3935d7c12900 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:13:17.846501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-af963a4c-ad77-46b3-a290-3935d7c12900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:13:17.877390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:17.914986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 658/1294] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:13:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:13:17.938144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646d77-9e8f-4253-8a0c-bb6905f5d931 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:17.969292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:18.038280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646d77-9e8f-4253-8a0c-bb6905f5d931 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:18.073656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fbf96-7184-40b7-9dbc-7ac317422cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:18.074223 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fbf96-7184-40b7-9dbc-7ac317422cf9 None admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:18.074514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fbf96-7184-40b7-9dbc-7ac317422cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:18.074793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fbf96-7184-40b7-9dbc-7ac317422cf9 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:18.076205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fbf96-7184-40b7-9dbc-7ac317422cf9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:18.077660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fbf96-7184-40b7-9dbc-7ac317422cf9 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:18.135258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 637/1295] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 28 06:13:17 2022] POST /identity/v3/services => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:13:20.202980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ba27-cdd3-49a2-a16c-c0e38c1ab27d None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:20.203516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ba27-cdd3-49a2-a16c-c0e38c1ab27d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:20.203516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ba27-cdd3-49a2-a16c-c0e38c1ab27d None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:20.204904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 659/1296] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:13:20.210122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37d03a-be72-45ee-875e-d02d074238e0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:20.210563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37d03a-be72-45ee-875e-d02d074238e0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:20.210658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37d03a-be72-45ee-875e-d02d074238e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:20.229656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0f37d03a-be72-45ee-875e-d02d074238e0 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:13:20.230849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0f37d03a-be72-45ee-875e-d02d074238e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:13:20.305837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 638/1297] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:13:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:13:20.329721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6ba27-cdd3-49a2-a16c-c0e38c1ab27d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:20.439820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6ba27-cdd3-49a2-a16c-c0e38c1ab27d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:20.474353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eeb716-1207-49b8-8a9f-ad3a9930770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:20.474906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eeb716-1207-49b8-8a9f-ad3a9930770e None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:20.475195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eeb716-1207-49b8-8a9f-ad3a9930770e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:20.475478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eeb716-1207-49b8-8a9f-ad3a9930770e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:20.476113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eeb716-1207-49b8-8a9f-ad3a9930770e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:20.477516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eeb716-1207-49b8-8a9f-ad3a9930770e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-77eeb716-1207-49b8-8a9f-ad3a9930770e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:20.486310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:20.487530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:20.488507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 660/1298] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 28 06:13:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:20.493939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37d03a-be72-45ee-875e-d02d074238e0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:20.512601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:20.596940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37d03a-be72-45ee-875e-d02d074238e0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:20.632035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518d237-76b9-476a-be53-24f5863131d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:20.632705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518d237-76b9-476a-be53-24f5863131d6 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:20.632994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518d237-76b9-476a-be53-24f5863131d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:20.633333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518d237-76b9-476a-be53-24f5863131d6 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:20.634027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518d237-76b9-476a-be53-24f5863131d6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:20.634335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518d237-76b9-476a-be53-24f5863131d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:20.635721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518d237-76b9-476a-be53-24f5863131d6 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:20.641541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 639/1299] 10.222.0.69 () {56 vars in 1116 bytes} [Thu Jul 28 06:13:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:20.647626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eeb716-1207-49b8-8a9f-ad3a9930770e None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:20.679755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:20.751556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eeb716-1207-49b8-8a9f-ad3a9930770e None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:20.784949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cfb34-c6d4-4758-9eb8-2ae71ee2f959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:20.785649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cfb34-c6d4-4758-9eb8-2ae71ee2f959 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:20.785938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cfb34-c6d4-4758-9eb8-2ae71ee2f959 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:20.786221 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cfb34-c6d4-4758-9eb8-2ae71ee2f959 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:20.786979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cfb34-c6d4-4758-9eb8-2ae71ee2f959 None admin] RBAC: Adding query filter params (interface=public, service_id=02901975c7d94a9689f98423d7d940a1, region_id=RegionOne) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:20.787295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cfb34-c6d4-4758-9eb8-2ae71ee2f959 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:20.788671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cfb34-c6d4-4758-9eb8-2ae71ee2f959 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:20.794336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 661/1300] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 28 06:13:20 2022] GET /identity/v3/endpoints?service_id=02901975c7d94a9689f98423d7d940a1&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:20.800128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518d237-76b9-476a-be53-24f5863131d6 None admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:20.884347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:20.902352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518d237-76b9-476a-be53-24f5863131d6 None admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:20.938218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4de17d-194c-405a-8369-b265bf5c7b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:20.938842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4de17d-194c-405a-8369-b265bf5c7b4c None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:20.939130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4de17d-194c-405a-8369-b265bf5c7b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:20.939409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4de17d-194c-405a-8369-b265bf5c7b4c None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:20.940068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4de17d-194c-405a-8369-b265bf5c7b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:20.941483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4de17d-194c-405a-8369-b265bf5c7b4c None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:20.948446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 640/1301] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 28 06:13:20 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:13:23.017597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289042b5-cc1d-4d2d-8571-7401e279613f None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:23.018316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289042b5-cc1d-4d2d-8571-7401e279613f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:23.018316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289042b5-cc1d-4d2d-8571-7401e279613f None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:23.019492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 662/1302] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:13:23.024656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2cc33-47f2-4b7d-b886-a18038e3169e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:23.029921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2cc33-47f2-4b7d-b886-a18038e3169e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:23.029921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2cc33-47f2-4b7d-b886-a18038e3169e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:23.045210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-beb2cc33-47f2-4b7d-b886-a18038e3169e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:13:23.046439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-beb2cc33-47f2-4b7d-b886-a18038e3169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:13:23.114094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 641/1303] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:13:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:13:23.138035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289042b5-cc1d-4d2d-8571-7401e279613f None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:23.235557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:23.242109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289042b5-cc1d-4d2d-8571-7401e279613f None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:23.275194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c818e50-8761-4372-91a8-8f1c21ddd1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:23.275774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c818e50-8761-4372-91a8-8f1c21ddd1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:23.276064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c818e50-8761-4372-91a8-8f1c21ddd1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:23.276344 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c818e50-8761-4372-91a8-8f1c21ddd1f5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:23.276987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c818e50-8761-4372-91a8-8f1c21ddd1f5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:23.279318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c818e50-8761-4372-91a8-8f1c21ddd1f5 None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application [None req-0c818e50-8761-4372-91a8-8f1c21ddd1f5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:13:23.288497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:13:23.289720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application with Lock( Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: ERROR keystone.server.flask.application Jul 28 06:13:23.291082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 663/1304] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 28 06:13:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:13:23.296370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2cc33-47f2-4b7d-b886-a18038e3169e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:23.376107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:23.399709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2cc33-47f2-4b7d-b886-a18038e3169e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:23.434796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d1522-8d23-4cd9-b8a5-37e89df1ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:23.435649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d1522-8d23-4cd9-b8a5-37e89df1ae94 None admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:23.436018 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d1522-8d23-4cd9-b8a5-37e89df1ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:23.436212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d1522-8d23-4cd9-b8a5-37e89df1ae94 None admin] PATH_INFO: `/v3/services` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:23.436895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d1522-8d23-4cd9-b8a5-37e89df1ae94 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:13:23.437203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d1522-8d23-4cd9-b8a5-37e89df1ae94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:23.438623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d1522-8d23-4cd9-b8a5-37e89df1ae94 None admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:23.443792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 642/1305] 10.222.0.69 () {56 vars in 1116 bytes} [Thu Jul 28 06:13:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:13:23.449678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c818e50-8761-4372-91a8-8f1c21ddd1f5 None admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:13:23.561538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c818e50-8761-4372-91a8-8f1c21ddd1f5 None admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:13:23.594821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfcf32-ee78-46c1-97fe-0abbcc78d00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:13:23.595446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfcf32-ee78-46c1-97fe-0abbcc78d00e None admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:13:23.595732 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfcf32-ee78-46c1-97fe-0abbcc78d00e None admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:13:23.596010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfcf32-ee78-46c1-97fe-0abbcc78d00e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:13:23.596723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfcf32-ee78-46c1-97fe-0abbcc78d00e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:13:23.598128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfcf32-ee78-46c1-97fe-0abbcc78d00e None admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:13:23.692284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 664/1306] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:13:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:13:33.440667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=30 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:13:33.687212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:38.672790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02d45d-a999-41ad-ab75-e0efcf0b952a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:38.673849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02d45d-a999-41ad-ab75-e0efcf0b952a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:38.674978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02d45d-a999-41ad-ab75-e0efcf0b952a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:38.682217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48838de3-001b-429c-ab68-768cc7796b7c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:38.683166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48838de3-001b-429c-ab68-768cc7796b7c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:38.683660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48838de3-001b-429c-ab68-768cc7796b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:38.688648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 643/1307] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:19:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:19:38.793131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-48838de3-001b-429c-ab68-768cc7796b7c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:19:38.794632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-48838de3-001b-429c-ab68-768cc7796b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:38.953400 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 665/1308] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:19:39.016920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304a1d0-151d-4fd9-ae86-d4b4262b23c4 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:39.018506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304a1d0-151d-4fd9-ae86-d4b4262b23c4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:39.019184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304a1d0-151d-4fd9-ae86-d4b4262b23c4 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:39.021482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 644/1309] 10.222.0.69 () {54 vars in 965 bytes} [Thu Jul 28 06:19:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:19:39.029500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111aee-411e-4c77-b499-4baa38702221 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:39.029886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111aee-411e-4c77-b499-4baa38702221 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:39.030054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111aee-411e-4c77-b499-4baa38702221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:39.151111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b2111aee-411e-4c77-b499-4baa38702221 None None] MFA Rules not processed for user `05574eace38f4b6d945c7b1927fdadd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:19:39.152230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b2111aee-411e-4c77-b499-4baa38702221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:39.176876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:39.252749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304a1d0-151d-4fd9-ae86-d4b4262b23c4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:19:39.254612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7304a1d0-151d-4fd9-ae86-d4b4262b23c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:39.263602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 666/1310] 10.222.0.69 () {58 vars in 1062 bytes} [Thu Jul 28 06:19:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:19:39.380471 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304a1d0-151d-4fd9-ae86-d4b4262b23c4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:19:39.469840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a725e9-eca1-4223-a5cb-75c3626e7dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:19:39.470389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a725e9-eca1-4223-a5cb-75c3626e7dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:39.470667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a725e9-eca1-4223-a5cb-75c3626e7dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:39.471269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a725e9-eca1-4223-a5cb-75c3626e7dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:39.472844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-82a725e9-eca1-4223-a5cb-75c3626e7dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:39.537082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:39.598429 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a725e9-eca1-4223-a5cb-75c3626e7dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:19:39.599900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a725e9-eca1-4223-a5cb-75c3626e7dc8 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:19:39.637708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 645/1311] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:19:42.023746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4a94-93e0-42c5-934c-71ccaef0a0ad None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:42.024290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4a94-93e0-42c5-934c-71ccaef0a0ad None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:42.024290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4a94-93e0-42c5-934c-71ccaef0a0ad None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:42.025688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 667/1312] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:19:42.030809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67165359-83be-471f-a735-052530a26bf9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:42.031231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67165359-83be-471f-a735-052530a26bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:42.031343 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67165359-83be-471f-a735-052530a26bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:42.104990 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-67165359-83be-471f-a735-052530a26bf9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:19:42.106220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-67165359-83be-471f-a735-052530a26bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:42.208157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 646/1313] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:19:42.247184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f4a94-93e0-42c5-934c-71ccaef0a0ad None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:19:42.248731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-998f4a94-93e0-42c5-934c-71ccaef0a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:42.358795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f4a94-93e0-42c5-934c-71ccaef0a0ad None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:19:42.434856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:42.437191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a7f8f-79bd-4e8a-ab98-0bf203f0d6a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:19:42.437848 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a7f8f-79bd-4e8a-ab98-0bf203f0d6a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:42.438138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a7f8f-79bd-4e8a-ab98-0bf203f0d6a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:42.438421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a7f8f-79bd-4e8a-ab98-0bf203f0d6a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:42.440156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-bf9a7f8f-79bd-4e8a-ab98-0bf203f0d6a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:42.561549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a7f8f-79bd-4e8a-ab98-0bf203f0d6a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:19:42.563027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a7f8f-79bd-4e8a-ab98-0bf203f0d6a3 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:19:42.583634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 668/1314] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:19:45.115871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abf996-35b6-488c-83b0-fe61770f0080 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:45.116412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abf996-35b6-488c-83b0-fe61770f0080 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:45.116412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abf996-35b6-488c-83b0-fe61770f0080 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:45.118103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 647/1315] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:19:45.123104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3450dd7-8873-4f96-a461-f08fc2402c8d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:45.123499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3450dd7-8873-4f96-a461-f08fc2402c8d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:45.123666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3450dd7-8873-4f96-a461-f08fc2402c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:45.188585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b3450dd7-8873-4f96-a461-f08fc2402c8d None None] MFA Rules not processed for user `05a7b57c5cfe4d368e6c200fc07ecf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:19:45.189783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b3450dd7-8873-4f96-a461-f08fc2402c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:45.242450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:45.288485 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 669/1316] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:19:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:19:45.327948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abf996-35b6-488c-83b0-fe61770f0080 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:19:45.329572 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-03abf996-35b6-488c-83b0-fe61770f0080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:45.339988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:45.435937 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abf996-35b6-488c-83b0-fe61770f0080 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:19:45.511479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516a81c-cf8e-4a55-9753-0f9e5f0de8d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:19:45.512109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516a81c-cf8e-4a55-9753-0f9e5f0de8d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:45.512397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516a81c-cf8e-4a55-9753-0f9e5f0de8d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:45.512677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516a81c-cf8e-4a55-9753-0f9e5f0de8d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:45.514477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1516a81c-cf8e-4a55-9753-0f9e5f0de8d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:45.579550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:45.625745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516a81c-cf8e-4a55-9753-0f9e5f0de8d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:19:45.628257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516a81c-cf8e-4a55-9753-0f9e5f0de8d9 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:19:45.647043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 648/1317] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:19:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:19:48.452292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbef9f-3777-4b1b-9b09-6d39ed972ebc None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:48.452823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbef9f-3777-4b1b-9b09-6d39ed972ebc None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:48.453034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbef9f-3777-4b1b-9b09-6d39ed972ebc None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:48.454263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 670/1318] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:19:48.459626 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87fc0-faaf-4870-9cae-7d6df0f50007 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:48.460047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87fc0-faaf-4870-9cae-7d6df0f50007 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:48.460181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87fc0-faaf-4870-9cae-7d6df0f50007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:48.527203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a1f87fc0-faaf-4870-9cae-7d6df0f50007 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:19:48.528412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a1f87fc0-faaf-4870-9cae-7d6df0f50007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:48.624069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 649/1319] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:19:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:19:48.662942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbef9f-3777-4b1b-9b09-6d39ed972ebc None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:19:48.664611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-afdbef9f-3777-4b1b-9b09-6d39ed972ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:48.769867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbef9f-3777-4b1b-9b09-6d39ed972ebc None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:19:48.842805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5d53e-0e85-4a08-aaa0-162164b160b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:19:48.843432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5d53e-0e85-4a08-aaa0-162164b160b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:48.843722 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5d53e-0e85-4a08-aaa0-162164b160b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:48.844004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5d53e-0e85-4a08-aaa0-162164b160b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:48.845840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6ea5d53e-0e85-4a08-aaa0-162164b160b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:48.853791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:48.959161 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5d53e-0e85-4a08-aaa0-162164b160b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:19:48.960623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5d53e-0e85-4a08-aaa0-162164b160b4 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:19:48.979576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 671/1320] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:19:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:19:52.012854 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6374b-5b52-4ca1-babc-663f9c30b2b8 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:52.013563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6374b-5b52-4ca1-babc-663f9c30b2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:52.013563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6374b-5b52-4ca1-babc-663f9c30b2b8 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:52.014816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 650/1321] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:19:52.020676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534c6be-2ffe-4879-82de-dd3b85b1b6d7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:52.021110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534c6be-2ffe-4879-82de-dd3b85b1b6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:52.021267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534c6be-2ffe-4879-82de-dd3b85b1b6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:52.087572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-c534c6be-2ffe-4879-82de-dd3b85b1b6d7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:19:52.088781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c534c6be-2ffe-4879-82de-dd3b85b1b6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:52.148218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:52.185861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 672/1322] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:19:52.222635 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6374b-5b52-4ca1-babc-663f9c30b2b8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:19:52.224205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d6c6374b-5b52-4ca1-babc-663f9c30b2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:52.244914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:52.338082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6374b-5b52-4ca1-babc-663f9c30b2b8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:19:52.426901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32cc24-3ed6-41c1-a6a6-296aa59918d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:19:52.427538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32cc24-3ed6-41c1-a6a6-296aa59918d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:52.430360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32cc24-3ed6-41c1-a6a6-296aa59918d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:52.430360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32cc24-3ed6-41c1-a6a6-296aa59918d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:52.430360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cc32cc24-3ed6-41c1-a6a6-296aa59918d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:52.510887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:52.552077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32cc24-3ed6-41c1-a6a6-296aa59918d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:19:52.553575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32cc24-3ed6-41c1-a6a6-296aa59918d5 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:19:52.572310 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 651/1323] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:19:54.650198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03045392-8366-464b-92a5-75157e4c643c None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:54.650728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03045392-8366-464b-92a5-75157e4c643c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:54.650825 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03045392-8366-464b-92a5-75157e4c643c None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:54.652241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 673/1324] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:19:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:19:54.658243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854dd36-7da4-4427-b22c-f844c24116a7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:54.658705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854dd36-7da4-4427-b22c-f844c24116a7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:54.658812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854dd36-7da4-4427-b22c-f844c24116a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:54.734836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a854dd36-7da4-4427-b22c-f844c24116a7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:19:54.736732 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a854dd36-7da4-4427-b22c-f844c24116a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:54.837220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 652/1325] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:19:54.874034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03045392-8366-464b-92a5-75157e4c643c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:19:54.875829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-03045392-8366-464b-92a5-75157e4c643c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:54.983928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03045392-8366-464b-92a5-75157e4c643c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:19:55.058430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286db7fa-57d0-4103-bc51-c6c1414a647c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:19:55.059055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286db7fa-57d0-4103-bc51-c6c1414a647c service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:55.059342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286db7fa-57d0-4103-bc51-c6c1414a647c service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:55.059619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286db7fa-57d0-4103-bc51-c6c1414a647c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:55.061259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-286db7fa-57d0-4103-bc51-c6c1414a647c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:55.080995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:55.177017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286db7fa-57d0-4103-bc51-c6c1414a647c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:19:55.178508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286db7fa-57d0-4103-bc51-c6c1414a647c service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:19:55.197683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 674/1326] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:19:57.280390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146e70b-f7e2-4eed-b85f-52ae92839071 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:57.281116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146e70b-f7e2-4eed-b85f-52ae92839071 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:57.281116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146e70b-f7e2-4eed-b85f-52ae92839071 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:57.282549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 653/1327] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:19:57.288538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816364-6360-4e31-a1a5-6e88b0be5b5a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:57.289004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816364-6360-4e31-a1a5-6e88b0be5b5a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:57.289097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816364-6360-4e31-a1a5-6e88b0be5b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:57.363434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-a6816364-6360-4e31-a1a5-6e88b0be5b5a None None] MFA Rules not processed for user `05a7b57c5cfe4d368e6c200fc07ecf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:19:57.364968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a6816364-6360-4e31-a1a5-6e88b0be5b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:57.437076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:57.471951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 675/1328] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:19:57 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:19:57.522231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02117891-926c-454f-ab16-7779a0d4377a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:57.522520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02117891-926c-454f-ab16-7779a0d4377a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:57.522803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02117891-926c-454f-ab16-7779a0d4377a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:57.524180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 654/1329] 10.222.0.69 () {54 vars in 965 bytes} [Thu Jul 28 06:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:19:57.530334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98344-f647-463a-843c-72f77afdcce8 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:57.530718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98344-f647-463a-843c-72f77afdcce8 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:57.530952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98344-f647-463a-843c-72f77afdcce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:57.612873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-59d98344-f647-463a-843c-72f77afdcce8 None None] MFA Rules not processed for user `05574eace38f4b6d945c7b1927fdadd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:19:57.614133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-59d98344-f647-463a-843c-72f77afdcce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:57.695083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 676/1330] 10.222.0.69 () {58 vars in 1062 bytes} [Thu Jul 28 06:19:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:19:57.704831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02117891-926c-454f-ab16-7779a0d4377a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:19:57.706178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-02117891-926c-454f-ab16-7779a0d4377a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:57.737995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:57.812973 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02117891-926c-454f-ab16-7779a0d4377a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:19:57.888287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab424a-f9cb-4cd4-9d2a-d65608e3b099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:19:57.888929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab424a-f9cb-4cd4-9d2a-d65608e3b099 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:19:57.889218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab424a-f9cb-4cd4-9d2a-d65608e3b099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:19:57.889529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab424a-f9cb-4cd4-9d2a-d65608e3b099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:19:57.891206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a5ab424a-f9cb-4cd4-9d2a-d65608e3b099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:19:57.971877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:19:58.005699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab424a-f9cb-4cd4-9d2a-d65608e3b099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:19:58.007182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab424a-f9cb-4cd4-9d2a-d65608e3b099 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:19:58.026051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 655/1331] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:19:57 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:00.873359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219ac9b-6404-44c8-8cae-92239ca8f8ed None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:00.873929 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219ac9b-6404-44c8-8cae-92239ca8f8ed None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:00.873929 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219ac9b-6404-44c8-8cae-92239ca8f8ed None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:00.875302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 677/1332] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:00.881170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc101a7-a43e-4f05-a3a4-c308c1325ffa None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:00.881624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc101a7-a43e-4f05-a3a4-c308c1325ffa None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:00.881792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc101a7-a43e-4f05-a3a4-c308c1325ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:00.952832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7fc101a7-a43e-4f05-a3a4-c308c1325ffa None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:00.955208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7fc101a7-a43e-4f05-a3a4-c308c1325ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:01.059262 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 656/1333] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:01.096619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219ac9b-6404-44c8-8cae-92239ca8f8ed None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:01.098219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b219ac9b-6404-44c8-8cae-92239ca8f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:01.168626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:01.203567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219ac9b-6404-44c8-8cae-92239ca8f8ed None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:01.276682 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f014a-ac0c-4842-84e3-d24aba928230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:01.277356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f014a-ac0c-4842-84e3-d24aba928230 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:01.277650 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f014a-ac0c-4842-84e3-d24aba928230 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:01.277927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f014a-ac0c-4842-84e3-d24aba928230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:01.279557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ca3f014a-ac0c-4842-84e3-d24aba928230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:01.399531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f014a-ac0c-4842-84e3-d24aba928230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:01.400986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f014a-ac0c-4842-84e3-d24aba928230 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:01.411308 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:01.423709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 678/1334] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:03.494830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198278a7-816f-46d8-a00c-99b96c36840d None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:03.495431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198278a7-816f-46d8-a00c-99b96c36840d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:03.495431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198278a7-816f-46d8-a00c-99b96c36840d None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:03.496788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 657/1335] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:03.502313 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4830-0ca6-4ac6-96d1-dc11862dc14a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:03.505709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4830-0ca6-4ac6-96d1-dc11862dc14a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:03.505709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4830-0ca6-4ac6-96d1-dc11862dc14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:03.569389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-1c2e4830-0ca6-4ac6-96d1-dc11862dc14a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:03.570610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1c2e4830-0ca6-4ac6-96d1-dc11862dc14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:03.665181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 679/1336] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:03.705666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198278a7-816f-46d8-a00c-99b96c36840d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:03.707222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-198278a7-816f-46d8-a00c-99b96c36840d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:03.747288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:03.811288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198278a7-816f-46d8-a00c-99b96c36840d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:03.885389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10e803-728f-4a53-8d9d-260edd497ae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:03.886007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10e803-728f-4a53-8d9d-260edd497ae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:03.886290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10e803-728f-4a53-8d9d-260edd497ae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:03.886567 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10e803-728f-4a53-8d9d-260edd497ae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:03.888201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1c10e803-728f-4a53-8d9d-260edd497ae6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:03.975499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:04.001995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10e803-728f-4a53-8d9d-260edd497ae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:04.003450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10e803-728f-4a53-8d9d-260edd497ae6 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:04.022040 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 658/1337] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:07.017570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78f7e6-ec0b-4612-9eb8-fda35dac5ec4 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:07.018128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78f7e6-ec0b-4612-9eb8-fda35dac5ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:07.018225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78f7e6-ec0b-4612-9eb8-fda35dac5ec4 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:07.019531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 680/1338] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:07.024907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb59f0-318d-4e23-96e3-752525124b88 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:07.025372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb59f0-318d-4e23-96e3-752525124b88 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:07.025488 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb59f0-318d-4e23-96e3-752525124b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:07.092108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-06bb59f0-318d-4e23-96e3-752525124b88 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:07.093350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-06bb59f0-318d-4e23-96e3-752525124b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:07.192394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 659/1339] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:07.228384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78f7e6-ec0b-4612-9eb8-fda35dac5ec4 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:07.230006 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7a78f7e6-ec0b-4612-9eb8-fda35dac5ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:07.309776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:07.339638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78f7e6-ec0b-4612-9eb8-fda35dac5ec4 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:07.416741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1f4ef-9d9c-4e15-8923-bf6e8d89e43d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:07.417412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1f4ef-9d9c-4e15-8923-bf6e8d89e43d service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:07.417704 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1f4ef-9d9c-4e15-8923-bf6e8d89e43d service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:07.417985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1f4ef-9d9c-4e15-8923-bf6e8d89e43d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:07.419637 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-dec1f4ef-9d9c-4e15-8923-bf6e8d89e43d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:07.532645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1f4ef-9d9c-4e15-8923-bf6e8d89e43d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:07.534242 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1f4ef-9d9c-4e15-8923-bf6e8d89e43d service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:07.553634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 681/1340] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:11.886863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e333019-3557-4c09-86d6-8c2cadaa4b57 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:11.887412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e333019-3557-4c09-86d6-8c2cadaa4b57 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:11.887412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e333019-3557-4c09-86d6-8c2cadaa4b57 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:11.888804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 660/1341] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:11.894702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a0fcd-12ba-4459-a4fa-d15e00671a80 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:11.895194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a0fcd-12ba-4459-a4fa-d15e00671a80 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:11.895292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a0fcd-12ba-4459-a4fa-d15e00671a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:11.901436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:11.970892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-fc8a0fcd-12ba-4459-a4fa-d15e00671a80 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:11.972228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-fc8a0fcd-12ba-4459-a4fa-d15e00671a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:12.080769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 682/1342] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:12.121122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e333019-3557-4c09-86d6-8c2cadaa4b57 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:12.122780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4e333019-3557-4c09-86d6-8c2cadaa4b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:12.176910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:12.234924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e333019-3557-4c09-86d6-8c2cadaa4b57 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:12.315446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551f348-1c84-4446-8dbc-cfd08aed0cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:12.316002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551f348-1c84-4446-8dbc-cfd08aed0cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:12.316292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551f348-1c84-4446-8dbc-cfd08aed0cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:12.316576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551f348-1c84-4446-8dbc-cfd08aed0cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:12.318337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8551f348-1c84-4446-8dbc-cfd08aed0cd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:12.424086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:12.441200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551f348-1c84-4446-8dbc-cfd08aed0cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:12.442692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551f348-1c84-4446-8dbc-cfd08aed0cd3 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:12.461577 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 661/1343] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:15.416520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337bf50-eb2e-4dec-bfc4-98522c5bc8fe None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:15.417118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337bf50-eb2e-4dec-bfc4-98522c5bc8fe None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:15.417118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337bf50-eb2e-4dec-bfc4-98522c5bc8fe None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:15.418523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 683/1344] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:15.424709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be52a-f6bc-4e27-9648-8b8d456a9e76 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:15.425110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be52a-f6bc-4e27-9648-8b8d456a9e76 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:15.425280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be52a-f6bc-4e27-9648-8b8d456a9e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:15.496099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f09be52a-f6bc-4e27-9648-8b8d456a9e76 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:15.497345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f09be52a-f6bc-4e27-9648-8b8d456a9e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:15.599565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 662/1345] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:15.637876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337bf50-eb2e-4dec-bfc4-98522c5bc8fe None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:15.639444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f337bf50-eb2e-4dec-bfc4-98522c5bc8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:15.738640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:15.756785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337bf50-eb2e-4dec-bfc4-98522c5bc8fe None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:15.835181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a0c38-9f12-47ad-879c-8f4a7c8ee757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:15.835816 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a0c38-9f12-47ad-879c-8f4a7c8ee757 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:15.836102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a0c38-9f12-47ad-879c-8f4a7c8ee757 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:15.836379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a0c38-9f12-47ad-879c-8f4a7c8ee757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:15.838114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-740a0c38-9f12-47ad-879c-8f4a7c8ee757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:15.952939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a0c38-9f12-47ad-879c-8f4a7c8ee757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:15.955024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a0c38-9f12-47ad-879c-8f4a7c8ee757 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:15.973127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 684/1346] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:20.253326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6661758-d474-413f-a1a2-720194ff2da9 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:20.253880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6661758-d474-413f-a1a2-720194ff2da9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:20.253880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6661758-d474-413f-a1a2-720194ff2da9 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:20.255219 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 663/1347] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:20.260627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f0549-82f2-4e98-8056-75f966d0b29c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:20.261095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f0549-82f2-4e98-8056-75f966d0b29c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:20.261213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f0549-82f2-4e98-8056-75f966d0b29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:20.278282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:20.354201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b33f0549-82f2-4e98-8056-75f966d0b29c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:20.355433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b33f0549-82f2-4e98-8056-75f966d0b29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:20.465705 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 685/1348] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:20:20.508629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6661758-d474-413f-a1a2-720194ff2da9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:20.511389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c6661758-d474-413f-a1a2-720194ff2da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:20.579004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:20.629257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6661758-d474-413f-a1a2-720194ff2da9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:20.706891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d5907-3975-43b7-aa6b-f6e556c803ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:20.707533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d5907-3975-43b7-aa6b-f6e556c803ef service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:20.707821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d5907-3975-43b7-aa6b-f6e556c803ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:20.708103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d5907-3975-43b7-aa6b-f6e556c803ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:20.709815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-617d5907-3975-43b7-aa6b-f6e556c803ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:20.826481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:20.830817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d5907-3975-43b7-aa6b-f6e556c803ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:20.832412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d5907-3975-43b7-aa6b-f6e556c803ef service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:20.852847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 664/1349] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:26.027338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677c11-dfcf-47a3-975e-a4a502d12847 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:26.028404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677c11-dfcf-47a3-975e-a4a502d12847 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:26.028948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677c11-dfcf-47a3-975e-a4a502d12847 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:26.041212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 686/1350] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:20:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:26.043454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf319814-7f88-4291-8330-1d5921a4f07b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:26.049329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf319814-7f88-4291-8330-1d5921a4f07b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:26.050073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf319814-7f88-4291-8330-1d5921a4f07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:26.154318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bf319814-7f88-4291-8330-1d5921a4f07b None None] MFA Rules not processed for user `244a8787df7641f2bac9b51213ddb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:26.155514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bf319814-7f88-4291-8330-1d5921a4f07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:26.280321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 665/1351] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:20:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:30.463530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=59 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:30.509451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d792d52-e08d-437f-b7a0-77264763f173 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:30.509806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d792d52-e08d-437f-b7a0-77264763f173 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:30.510015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d792d52-e08d-437f-b7a0-77264763f173 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:30.511900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 687/1352] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:30.517569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4efdfe-f041-4c41-a30a-ccf703b31935 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:30.518077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4efdfe-f041-4c41-a30a-ccf703b31935 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:30.518161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4efdfe-f041-4c41-a30a-ccf703b31935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:30.575509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:30.601317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ef4efdfe-f041-4c41-a30a-ccf703b31935 None None] MFA Rules not processed for user `244a8787df7641f2bac9b51213ddb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:30.602718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ef4efdfe-f041-4c41-a30a-ccf703b31935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:30.728295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 666/1353] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:20:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:31.974449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8b68-9ee8-42be-a40a-72b43523158e None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:31.974955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8b68-9ee8-42be-a40a-72b43523158e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:31.974955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8b68-9ee8-42be-a40a-72b43523158e None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:31.976278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 688/1354] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:31.981255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd018c4-b70b-4899-8b3b-6e6eb2ae182b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:31.981689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd018c4-b70b-4899-8b3b-6e6eb2ae182b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:31.982081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd018c4-b70b-4899-8b3b-6e6eb2ae182b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:32.061025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-afd018c4-b70b-4899-8b3b-6e6eb2ae182b None None] MFA Rules not processed for user `244a8787df7641f2bac9b51213ddb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:32.062939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-afd018c4-b70b-4899-8b3b-6e6eb2ae182b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:32.105777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:32.173508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 667/1355] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:20:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:35.696559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83ca50-ce68-4ab8-b82d-0ee66a523884 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:35.697135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83ca50-ce68-4ab8-b82d-0ee66a523884 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:35.697135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83ca50-ce68-4ab8-b82d-0ee66a523884 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:35.698679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 689/1356] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:35.705210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dedd8d-6d18-4487-a063-c04720c95e2a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:35.705756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dedd8d-6d18-4487-a063-c04720c95e2a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:35.705964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dedd8d-6d18-4487-a063-c04720c95e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:35.797371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-21dedd8d-6d18-4487-a063-c04720c95e2a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:35.798115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-21dedd8d-6d18-4487-a063-c04720c95e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:35.920024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 668/1357] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:35.974206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dfdde-8626-4448-9400-c5afc3a5401e None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:35.974811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dfdde-8626-4448-9400-c5afc3a5401e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:35.975265 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dfdde-8626-4448-9400-c5afc3a5401e None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:35.977015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 690/1358] 10.222.0.69 () {54 vars in 962 bytes} [Thu Jul 28 06:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:35.986074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852e3ae-c5b4-433a-bd96-f80bfcaa1dd0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:35.989809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852e3ae-c5b4-433a-bd96-f80bfcaa1dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:35.989809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852e3ae-c5b4-433a-bd96-f80bfcaa1dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:36.005888 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:36.075499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3852e3ae-c5b4-433a-bd96-f80bfcaa1dd0 None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:36.077136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3852e3ae-c5b4-433a-bd96-f80bfcaa1dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:36.225267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 669/1359] 10.222.0.69 () {58 vars in 1059 bytes} [Thu Jul 28 06:20:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:20:36.233062 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dfdde-8626-4448-9400-c5afc3a5401e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:36.235383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-633dfdde-8626-4448-9400-c5afc3a5401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:36.400796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dfdde-8626-4448-9400-c5afc3a5401e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:36.485867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:36.529624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24678ea-ce52-4fbd-b336-d3425aca9bb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:36.539262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24678ea-ce52-4fbd-b336-d3425aca9bb5 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:36.539262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24678ea-ce52-4fbd-b336-d3425aca9bb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:36.539262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24678ea-ce52-4fbd-b336-d3425aca9bb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:36.539262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f24678ea-ce52-4fbd-b336-d3425aca9bb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:36.634933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24678ea-ce52-4fbd-b336-d3425aca9bb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:36.636552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24678ea-ce52-4fbd-b336-d3425aca9bb5 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:36.656567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 691/1360] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:38.901412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d28de-5d1e-414f-98e4-3f642f5e23ff None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:38.902020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d28de-5d1e-414f-98e4-3f642f5e23ff None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:38.902020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d28de-5d1e-414f-98e4-3f642f5e23ff None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:38.903473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 670/1361] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:38.909200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efcb89-ca6d-4028-a6b6-17ef035d8fc6 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:38.909753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efcb89-ca6d-4028-a6b6-17ef035d8fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:38.909859 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efcb89-ca6d-4028-a6b6-17ef035d8fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:38.988310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e0efcb89-ca6d-4028-a6b6-17ef035d8fc6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:38.989634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e0efcb89-ca6d-4028-a6b6-17ef035d8fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:39.030326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:39.096740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 692/1362] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:39.147329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d28de-5d1e-414f-98e4-3f642f5e23ff None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:39.148943 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-350d28de-5d1e-414f-98e4-3f642f5e23ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:39.223603 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:39.327681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d28de-5d1e-414f-98e4-3f642f5e23ff None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:39.498307 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab40a5-b99e-4c18-93a3-4c45c8edb0c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:39.498839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab40a5-b99e-4c18-93a3-4c45c8edb0c3 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:39.499190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab40a5-b99e-4c18-93a3-4c45c8edb0c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:39.500100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab40a5-b99e-4c18-93a3-4c45c8edb0c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:39.502006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-68ab40a5-b99e-4c18-93a3-4c45c8edb0c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:39.529763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:39.605697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab40a5-b99e-4c18-93a3-4c45c8edb0c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:39.607356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab40a5-b99e-4c18-93a3-4c45c8edb0c3 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:39.627761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 671/1363] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:42.132581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8924cb-6b61-4cac-9974-ba83bd95e604 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:42.133147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8924cb-6b61-4cac-9974-ba83bd95e604 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:42.133147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8924cb-6b61-4cac-9974-ba83bd95e604 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:42.134886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 693/1364] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:42.139963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e1e82-1a32-4347-b5dd-117a7a11f0e1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:42.140452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e1e82-1a32-4347-b5dd-117a7a11f0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:42.140538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e1e82-1a32-4347-b5dd-117a7a11f0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:42.221258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-881e1e82-1a32-4347-b5dd-117a7a11f0e1 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:42.222519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-881e1e82-1a32-4347-b5dd-117a7a11f0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:42.335284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:42.338950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 672/1365] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:42.387375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15f755-b187-447a-94b9-cd958e8806a2 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:42.387854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15f755-b187-447a-94b9-cd958e8806a2 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:42.388249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15f755-b187-447a-94b9-cd958e8806a2 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:42.390501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 694/1366] 10.222.0.69 () {54 vars in 962 bytes} [Thu Jul 28 06:20:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:42.399697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0d9d7-5552-4d42-9331-3f990e65db04 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:42.399697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0d9d7-5552-4d42-9331-3f990e65db04 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:42.399697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0d9d7-5552-4d42-9331-3f990e65db04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:42.484988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1dc0d9d7-5552-4d42-9331-3f990e65db04 None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:42.486372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1dc0d9d7-5552-4d42-9331-3f990e65db04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:42.621545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 673/1367] 10.222.0.69 () {58 vars in 1059 bytes} [Thu Jul 28 06:20:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:42.629918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15f755-b187-447a-94b9-cd958e8806a2 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:42.631457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7e15f755-b187-447a-94b9-cd958e8806a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:42.790394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15f755-b187-447a-94b9-cd958e8806a2 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:42.812406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:42.914988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2c754-2d20-4fc1-a4ef-1d1ec70fd532 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:42.915616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2c754-2d20-4fc1-a4ef-1d1ec70fd532 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:42.915906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2c754-2d20-4fc1-a4ef-1d1ec70fd532 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:42.916187 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2c754-2d20-4fc1-a4ef-1d1ec70fd532 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:42.917868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-cec2c754-2d20-4fc1-a4ef-1d1ec70fd532 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:43.016462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2c754-2d20-4fc1-a4ef-1d1ec70fd532 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:43.018131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2c754-2d20-4fc1-a4ef-1d1ec70fd532 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:43.037809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 695/1368] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:43.987914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2a8ae-615e-424d-b3dd-710cad23259a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:43.988565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2a8ae-615e-424d-b3dd-710cad23259a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:43.988565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2a8ae-615e-424d-b3dd-710cad23259a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:43.990807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 674/1369] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:43.995943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034794b3-7472-4543-9a65-e12b8aa76859 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:43.996466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034794b3-7472-4543-9a65-e12b8aa76859 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:43.996548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034794b3-7472-4543-9a65-e12b8aa76859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:44.032471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:44.071554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-034794b3-7472-4543-9a65-e12b8aa76859 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:44.072863 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-034794b3-7472-4543-9a65-e12b8aa76859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:44.178803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 696/1370] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:44.533716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ffb67-52e3-420a-b6d8-6fad7c26838d None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:44.533716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ffb67-52e3-420a-b6d8-6fad7c26838d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:44.533716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ffb67-52e3-420a-b6d8-6fad7c26838d None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:44.533716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 675/1371] 10.222.0.69 () {54 vars in 964 bytes} [Thu Jul 28 06:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:44.538587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6978a-298d-437e-b8ae-e8ccfcf959ae None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:44.538881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6978a-298d-437e-b8ae-e8ccfcf959ae None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:44.541536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6978a-298d-437e-b8ae-e8ccfcf959ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:44.621765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f7e6978a-298d-437e-b8ae-e8ccfcf959ae None None] MFA Rules not processed for user `cbbbf34c238d44db8588d56f35827390`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:44.622907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f7e6978a-298d-437e-b8ae-e8ccfcf959ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:44.675320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:44.707571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 697/1372] 10.222.0.69 () {58 vars in 1061 bytes} [Thu Jul 28 06:20:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:44.715992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ffb67-52e3-420a-b6d8-6fad7c26838d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:44.717590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7f3ffb67-52e3-420a-b6d8-6fad7c26838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:44.777799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:44.836335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ffb67-52e3-420a-b6d8-6fad7c26838d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:44.926081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f4439-502c-4e27-8094-3408fbd0b2dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:44.926640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4439-502c-4e27-8094-3408fbd0b2dc service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:44.926925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4439-502c-4e27-8094-3408fbd0b2dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:44.927207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4439-502c-4e27-8094-3408fbd0b2dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:44.928938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e83f4439-502c-4e27-8094-3408fbd0b2dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:45.036961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:45.051805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f4439-502c-4e27-8094-3408fbd0b2dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:45.053281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f4439-502c-4e27-8094-3408fbd0b2dc service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:45.072740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 676/1373] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:20:45.367150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2432fb-d43a-45aa-93de-c0a140a98145 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:45.367150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2432fb-d43a-45aa-93de-c0a140a98145 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:45.367150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2432fb-d43a-45aa-93de-c0a140a98145 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:45.367150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 698/1374] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:45.368277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984e531-2d31-441e-a41a-1250dcc35dd0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:45.368277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984e531-2d31-441e-a41a-1250dcc35dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:45.368277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984e531-2d31-441e-a41a-1250dcc35dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:45.438151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f984e531-2d31-441e-a41a-1250dcc35dd0 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:45.439241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f984e531-2d31-441e-a41a-1250dcc35dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:45.557786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 677/1375] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:45.606944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2432fb-d43a-45aa-93de-c0a140a98145 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:45.609084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7f2432fb-d43a-45aa-93de-c0a140a98145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:45.764576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2432fb-d43a-45aa-93de-c0a140a98145 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:45.816616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:45.881123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec3688-a3ed-4155-ab45-8f393a022563 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:45.881793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec3688-a3ed-4155-ab45-8f393a022563 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:45.885217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec3688-a3ed-4155-ab45-8f393a022563 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:45.885217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec3688-a3ed-4155-ab45-8f393a022563 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:45.885217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8cec3688-a3ed-4155-ab45-8f393a022563 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:45.999921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec3688-a3ed-4155-ab45-8f393a022563 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:46.001584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec3688-a3ed-4155-ab45-8f393a022563 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:46.021683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 699/1376] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:48.234717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d57a25-e9d4-4fdf-b77b-5527ce44194c None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:48.235251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d57a25-e9d4-4fdf-b77b-5527ce44194c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:48.235251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d57a25-e9d4-4fdf-b77b-5527ce44194c None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:48.236636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 678/1377] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:48.242340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82e9a5-f46c-4c9f-b193-b368c161bfb8 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:48.242784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82e9a5-f46c-4c9f-b193-b368c161bfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:48.242884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82e9a5-f46c-4c9f-b193-b368c161bfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:48.337872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-9e82e9a5-f46c-4c9f-b193-b368c161bfb8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:48.339544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9e82e9a5-f46c-4c9f-b193-b368c161bfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:48.349892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:48.455684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 700/1378] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:48.505983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d57a25-e9d4-4fdf-b77b-5527ce44194c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:48.508632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-52d57a25-e9d4-4fdf-b77b-5527ce44194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:48.578252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:48.696111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d57a25-e9d4-4fdf-b77b-5527ce44194c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:48.842333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e5193-1058-4b47-9d80-648353ed6853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:48.845566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e5193-1058-4b47-9d80-648353ed6853 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:48.845566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e5193-1058-4b47-9d80-648353ed6853 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:48.845566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e5193-1058-4b47-9d80-648353ed6853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:48.847305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-320e5193-1058-4b47-9d80-648353ed6853 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:48.870024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:48.952757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e5193-1058-4b47-9d80-648353ed6853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:48.954579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e5193-1058-4b47-9d80-648353ed6853 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:48.974681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 679/1379] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:51.156616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d15667-b4de-4ed5-97bd-79ab38c597c6 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:51.157180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d15667-b4de-4ed5-97bd-79ab38c597c6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:51.157180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d15667-b4de-4ed5-97bd-79ab38c597c6 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:51.158759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 701/1380] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:51.163914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e937a7-bbaa-4b58-bc15-5035c71c099e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:51.164590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e937a7-bbaa-4b58-bc15-5035c71c099e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:51.164590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e937a7-bbaa-4b58-bc15-5035c71c099e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:51.242023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-44e937a7-bbaa-4b58-bc15-5035c71c099e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:51.243326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-44e937a7-bbaa-4b58-bc15-5035c71c099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:51.342220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:51.354208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 680/1381] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:51.400114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d15667-b4de-4ed5-97bd-79ab38c597c6 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:51.401890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-04d15667-b4de-4ed5-97bd-79ab38c597c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:51.556205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:51.568613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d15667-b4de-4ed5-97bd-79ab38c597c6 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:51.706749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57dedc-9293-4ef2-ad49-d90494f33768 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:51.707864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57dedc-9293-4ef2-ad49-d90494f33768 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:51.708155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57dedc-9293-4ef2-ad49-d90494f33768 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:51.708444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57dedc-9293-4ef2-ad49-d90494f33768 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:51.710358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0e57dedc-9293-4ef2-ad49-d90494f33768 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:51.820533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57dedc-9293-4ef2-ad49-d90494f33768 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:51.825185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57dedc-9293-4ef2-ad49-d90494f33768 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:51.839194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:51.848236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 702/1382] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:54.118770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01ed77-a71c-4f5e-acf1-b808aa75aec9 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:54.119346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01ed77-a71c-4f5e-acf1-b808aa75aec9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:54.119346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01ed77-a71c-4f5e-acf1-b808aa75aec9 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:54.120697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 681/1383] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:20:54.126853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52942a9d-e7f0-441f-825e-bf169383d822 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:54.127330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52942a9d-e7f0-441f-825e-bf169383d822 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:54.127420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52942a9d-e7f0-441f-825e-bf169383d822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:54.200595 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-52942a9d-e7f0-441f-825e-bf169383d822 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:54.202469 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-52942a9d-e7f0-441f-825e-bf169383d822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:54.311815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 703/1384] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:54.355952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01ed77-a71c-4f5e-acf1-b808aa75aec9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:54.357875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ae01ed77-a71c-4f5e-acf1-b808aa75aec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:54.532711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01ed77-a71c-4f5e-acf1-b808aa75aec9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:54.611609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:54.664046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af7906-f630-4bee-a8c9-f1a0652002a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:54.664680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7906-f630-4bee-a8c9-f1a0652002a7 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:54.664967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7906-f630-4bee-a8c9-f1a0652002a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:54.665252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7906-f630-4bee-a8c9-f1a0652002a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:54.666963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e1af7906-f630-4bee-a8c9-f1a0652002a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:54.767327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af7906-f630-4bee-a8c9-f1a0652002a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:54.769030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af7906-f630-4bee-a8c9-f1a0652002a7 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:54.791719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 682/1385] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:20:57.185746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b4f6f-3f26-489f-b03d-e22da1d328cc None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:57.186787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b4f6f-3f26-489f-b03d-e22da1d328cc None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:57.186963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b4f6f-3f26-489f-b03d-e22da1d328cc None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:57.188351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 704/1386] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:20:57.194987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9229-56de-4c02-a08c-a7e3fa052960 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:57.195452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9229-56de-4c02-a08c-a7e3fa052960 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:57.195533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9229-56de-4c02-a08c-a7e3fa052960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:57.272127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6c3a9229-56de-4c02-a08c-a7e3fa052960 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:20:57.273473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6c3a9229-56de-4c02-a08c-a7e3fa052960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:57.314727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:57.403745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 683/1387] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:20:57.476731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b4f6f-3f26-489f-b03d-e22da1d328cc None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:20:57.478353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ce3b4f6f-3f26-489f-b03d-e22da1d328cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:57.594022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:57.677599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b4f6f-3f26-489f-b03d-e22da1d328cc None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:20:57.815794 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ff15a-71be-4229-9559-353cc5ae3a9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:20:57.816434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ff15a-71be-4229-9559-353cc5ae3a9d service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:20:57.816725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ff15a-71be-4229-9559-353cc5ae3a9d service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:20:57.817013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ff15a-71be-4229-9559-353cc5ae3a9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:20:57.818722 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-073ff15a-71be-4229-9559-353cc5ae3a9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:20:57.863875 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:20:57.918222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ff15a-71be-4229-9559-353cc5ae3a9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:20:57.919867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ff15a-71be-4229-9559-353cc5ae3a9d service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:20:57.941009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 705/1388] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:20:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:00.156696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a00ad9-4aa8-4d7f-b8b8-4de3ac0d6c49 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:00.157272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a00ad9-4aa8-4d7f-b8b8-4de3ac0d6c49 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:00.157272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a00ad9-4aa8-4d7f-b8b8-4de3ac0d6c49 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:00.158640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 684/1389] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:00.164026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547b01-01b5-48a8-b0c8-cfb173923298 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:00.164453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547b01-01b5-48a8-b0c8-cfb173923298 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:00.164600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547b01-01b5-48a8-b0c8-cfb173923298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:00.235561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-62547b01-01b5-48a8-b0c8-cfb173923298 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:00.236790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-62547b01-01b5-48a8-b0c8-cfb173923298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:00.335622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 706/1390] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:00.381698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a00ad9-4aa8-4d7f-b8b8-4de3ac0d6c49 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:00.383332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-83a00ad9-4aa8-4d7f-b8b8-4de3ac0d6c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:00.540737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a00ad9-4aa8-4d7f-b8b8-4de3ac0d6c49 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:00.558829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:00.607052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638dbe1-cdca-4d1e-8c40-e054ca690b20 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:00.607403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638dbe1-cdca-4d1e-8c40-e054ca690b20 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:00.608210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638dbe1-cdca-4d1e-8c40-e054ca690b20 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:00.609789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 707/1391] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:21:00.615095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ab6a0-e7d4-4c75-b80b-8fd8fef3b8d6 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:00.615414 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ab6a0-e7d4-4c75-b80b-8fd8fef3b8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:00.615811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ab6a0-e7d4-4c75-b80b-8fd8fef3b8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:00.631745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:00.659160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb683439-f7d6-4f0f-8c95-8c17fe213bdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:00.659836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb683439-f7d6-4f0f-8c95-8c17fe213bdf service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:00.660122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb683439-f7d6-4f0f-8c95-8c17fe213bdf service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:00.660399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb683439-f7d6-4f0f-8c95-8c17fe213bdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:00.662090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cb683439-f7d6-4f0f-8c95-8c17fe213bdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:00.684156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-cc1ab6a0-e7d4-4c75-b80b-8fd8fef3b8d6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:00.685377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-cc1ab6a0-e7d4-4c75-b80b-8fd8fef3b8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:00.755828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb683439-f7d6-4f0f-8c95-8c17fe213bdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:00.757475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb683439-f7d6-4f0f-8c95-8c17fe213bdf service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:00.776402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 685/1392] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:00.786494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 708/1393] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:21:00.842523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a332ed-f446-446a-a107-15e4560a8252 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:00.842832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a332ed-f446-446a-a107-15e4560a8252 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:00.843284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a332ed-f446-446a-a107-15e4560a8252 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:00.844678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 686/1394] 10.222.0.69 () {54 vars in 966 bytes} [Thu Jul 28 06:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:00.849900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55b14-fe18-42d7-bd01-67781768e42b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:00.850225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55b14-fe18-42d7-bd01-67781768e42b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:00.850507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55b14-fe18-42d7-bd01-67781768e42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:00.929569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-19a55b14-fe18-42d7-bd01-67781768e42b None None] MFA Rules not processed for user `cc4fe04b11ec412e8f4eb67c6e54b8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:00.930799 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-19a55b14-fe18-42d7-bd01-67781768e42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:00.965459 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:01.012522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 709/1395] 10.222.0.69 () {58 vars in 1063 bytes} [Thu Jul 28 06:21:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:01.020714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a332ed-f446-446a-a107-15e4560a8252 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:01.022303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d6a332ed-f446-446a-a107-15e4560a8252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:01.049379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:01.143400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a332ed-f446-446a-a107-15e4560a8252 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:01.220230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bd6ed-2b32-4486-8ff8-7230931086ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4fe04b11ec412e8f4eb67c6e54b8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:01.220895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd6ed-2b32-4486-8ff8-7230931086ce service glance] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:01.221137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd6ed-2b32-4486-8ff8-7230931086ce service glance] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:01.221464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd6ed-2b32-4486-8ff8-7230931086ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:01.223181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bf3bd6ed-2b32-4486-8ff8-7230931086ce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:01.299384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:01.340070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bd6ed-2b32-4486-8ff8-7230931086ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:01.341748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bd6ed-2b32-4486-8ff8-7230931086ce service glance] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:01.363505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 687/1396] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:02.257553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b800b9c-b7f1-4a78-b0b4-9d5b86a59b6d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:02.258066 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b800b9c-b7f1-4a78-b0b4-9d5b86a59b6d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:02.258251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b800b9c-b7f1-4a78-b0b4-9d5b86a59b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:02.326306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4b800b9c-b7f1-4a78-b0b4-9d5b86a59b6d None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:02.327532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4b800b9c-b7f1-4a78-b0b4-9d5b86a59b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:02.446687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 710/1397] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:21:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:02.493514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab3ce4-c973-4aba-88ba-2b0ca3768c33 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:02.494039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab3ce4-c973-4aba-88ba-2b0ca3768c33 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:02.494151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab3ce4-c973-4aba-88ba-2b0ca3768c33 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:02.495681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 688/1398] 10.222.0.69 () {54 vars in 943 bytes} [Thu Jul 28 06:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:02.502415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37f9b3-2d16-4430-8eda-10a02cb66bdc None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:02.502697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37f9b3-2d16-4430-8eda-10a02cb66bdc None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:02.502987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37f9b3-2d16-4430-8eda-10a02cb66bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:02.563625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:02.584397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-fa37f9b3-2d16-4430-8eda-10a02cb66bdc None None] MFA Rules not processed for user `77571de30e934d6590ff7b58a000f5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:02.585693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-fa37f9b3-2d16-4430-8eda-10a02cb66bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:02.695942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 711/1399] 10.222.0.69 () {58 vars in 1040 bytes} [Thu Jul 28 06:21:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:02.706191 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab3ce4-c973-4aba-88ba-2b0ca3768c33 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:02.707747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d5ab3ce4-c973-4aba-88ba-2b0ca3768c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:02.852057 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab3ce4-c973-4aba-88ba-2b0ca3768c33 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:02.896357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:02.963935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e907c7-d2df-45e2-b8aa-0a16ec4e59ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:02.964570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e907c7-d2df-45e2-b8aa-0a16ec4e59ab service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:02.964856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e907c7-d2df-45e2-b8aa-0a16ec4e59ab service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:02.965139 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e907c7-d2df-45e2-b8aa-0a16ec4e59ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:02.966873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-78e907c7-d2df-45e2-b8aa-0a16ec4e59ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:02.996826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa800c-5418-4403-a42e-4292c4166f74 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:02.997144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa800c-5418-4403-a42e-4292c4166f74 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:02.997500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa800c-5418-4403-a42e-4292c4166f74 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:02.999503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 712/1400] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:21:03.004658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86d40c-77b5-439c-b21a-a068f76c88d3 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:03.004943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86d40c-77b5-439c-b21a-a068f76c88d3 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:03.005217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86d40c-77b5-439c-b21a-a068f76c88d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:03.052007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e907c7-d2df-45e2-b8aa-0a16ec4e59ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:03.053667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e907c7-d2df-45e2-b8aa-0a16ec4e59ab service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:03.062477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 689/1401] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:21:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:21:03.074881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-0a86d40c-77b5-439c-b21a-a068f76c88d3 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:03.076117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0a86d40c-77b5-439c-b21a-a068f76c88d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:03.135345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:03.178011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 713/1402] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:21:03.223863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e907c7-d2df-45e2-b8aa-0a16ec4e59ab service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:03.225106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-78e907c7-d2df-45e2-b8aa-0a16ec4e59ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:03.294739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:03.318651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e907c7-d2df-45e2-b8aa-0a16ec4e59ab service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:03.441510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9fb9c-41f8-40e1-bbdc-d6ad5449e12e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:03.442112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9fb9c-41f8-40e1-bbdc-d6ad5449e12e service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:03.442414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9fb9c-41f8-40e1-bbdc-d6ad5449e12e service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:03.442673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9fb9c-41f8-40e1-bbdc-d6ad5449e12e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:03.444311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-29a9fb9c-41f8-40e1-bbdc-d6ad5449e12e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:03.547923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9fb9c-41f8-40e1-bbdc-d6ad5449e12e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:03.549742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9fb9c-41f8-40e1-bbdc-d6ad5449e12e service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:03.555405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:03.570582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 690/1403] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:05.827298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10f842-0d0c-4eeb-a8d2-b7b6249d02c2 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:05.827913 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10f842-0d0c-4eeb-a8d2-b7b6249d02c2 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:05.827913 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10f842-0d0c-4eeb-a8d2-b7b6249d02c2 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:05.829387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 714/1404] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:05.834844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb88f0-b290-4457-af3f-f75c66ab1386 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:05.835844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb88f0-b290-4457-af3f-f75c66ab1386 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:05.836192 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb88f0-b290-4457-af3f-f75c66ab1386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:05.916911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9ddb88f0-b290-4457-af3f-f75c66ab1386 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:05.918136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9ddb88f0-b290-4457-af3f-f75c66ab1386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:06.033582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 691/1405] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:06.078525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10f842-0d0c-4eeb-a8d2-b7b6249d02c2 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:06.080151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8c10f842-0d0c-4eeb-a8d2-b7b6249d02c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:06.260492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10f842-0d0c-4eeb-a8d2-b7b6249d02c2 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:06.317854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:06.412515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe7fa6-b522-472b-858d-1e2aafc1824c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:06.413287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe7fa6-b522-472b-858d-1e2aafc1824c service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:06.413672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe7fa6-b522-472b-858d-1e2aafc1824c service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:06.414029 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe7fa6-b522-472b-858d-1e2aafc1824c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:06.415909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e3fe7fa6-b522-472b-858d-1e2aafc1824c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:06.538274 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe7fa6-b522-472b-858d-1e2aafc1824c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:06.540654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe7fa6-b522-472b-858d-1e2aafc1824c service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:06.562446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 715/1406] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:09.037725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549bbc9-a044-4eb7-a251-46cf4e138c78 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:09.038300 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549bbc9-a044-4eb7-a251-46cf4e138c78 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:09.038734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549bbc9-a044-4eb7-a251-46cf4e138c78 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:09.045529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 692/1407] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:21:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:09.048410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d134cb-da0b-4093-96b2-a72557f3e4ee None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:09.048956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d134cb-da0b-4093-96b2-a72557f3e4ee None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:09.049216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d134cb-da0b-4093-96b2-a72557f3e4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:09.140035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:09.148749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-95d134cb-da0b-4093-96b2-a72557f3e4ee None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:09.150399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-95d134cb-da0b-4093-96b2-a72557f3e4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:09.263340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 716/1408] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:09.315283 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549bbc9-a044-4eb7-a251-46cf4e138c78 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:09.316881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b549bbc9-a044-4eb7-a251-46cf4e138c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:09.440096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:09.519108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549bbc9-a044-4eb7-a251-46cf4e138c78 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:09.660157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e8910-90be-4410-b2e7-2d9be91f33aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:09.660157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e8910-90be-4410-b2e7-2d9be91f33aa service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:09.660157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e8910-90be-4410-b2e7-2d9be91f33aa service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:09.660157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e8910-90be-4410-b2e7-2d9be91f33aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:09.662312 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-097e8910-90be-4410-b2e7-2d9be91f33aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:09.706989 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:09.788525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e8910-90be-4410-b2e7-2d9be91f33aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:09.790188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e8910-90be-4410-b2e7-2d9be91f33aa service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:09.822263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 693/1409] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:19.261341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=44 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:20.232472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=31 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:26.733511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456d11-958f-4443-8f71-d4dd6eedb644 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:26.734064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456d11-958f-4443-8f71-d4dd6eedb644 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:26.734064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456d11-958f-4443-8f71-d4dd6eedb644 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:26.735388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 717/1410] 10.222.0.69 () {54 vars in 918 bytes} [Thu Jul 28 06:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:26.741041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18a5b8-9938-4a38-991a-84c97d721ec5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:26.741508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18a5b8-9938-4a38-991a-84c97d721ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:26.741620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18a5b8-9938-4a38-991a-84c97d721ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:26.818272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6c18a5b8-9938-4a38-991a-84c97d721ec5 None None] MFA Rules not processed for user `244a8787df7641f2bac9b51213ddb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:26.819494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6c18a5b8-9938-4a38-991a-84c97d721ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:26.925837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 694/1411] 10.222.0.69 () {58 vars in 1015 bytes} [Thu Jul 28 06:21:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:26.940494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f2994-cbcf-4884-9047-56cde516f6cc None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:26.940782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f2994-cbcf-4884-9047-56cde516f6cc None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:26.941067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f2994-cbcf-4884-9047-56cde516f6cc None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:26.942494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 718/1412] 10.222.0.69 () {54 vars in 918 bytes} [Thu Jul 28 06:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:26.947224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b350d-6495-4baa-a932-ecb22611bb7e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:26.947517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b350d-6495-4baa-a932-ecb22611bb7e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:26.947852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b350d-6495-4baa-a932-ecb22611bb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:27.021890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:27.025001 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b93b350d-6495-4baa-a932-ecb22611bb7e None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:27.026239 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b93b350d-6495-4baa-a932-ecb22611bb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:27.158342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 695/1413] 10.222.0.69 () {58 vars in 1015 bytes} [Thu Jul 28 06:21:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:21:29.652555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506696d-dbc3-4112-967a-a5324667ef10 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:29.653087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506696d-dbc3-4112-967a-a5324667ef10 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:29.653087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506696d-dbc3-4112-967a-a5324667ef10 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:29.654539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:29.659830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13908c77-45b1-4069-9d55-c7d40cd7484c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:29.660262 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13908c77-45b1-4069-9d55-c7d40cd7484c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:29.660391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13908c77-45b1-4069-9d55-c7d40cd7484c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:29.726051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-13908c77-45b1-4069-9d55-c7d40cd7484c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:29.727250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-13908c77-45b1-4069-9d55-c7d40cd7484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:29.783221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:29.827988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:29.871340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506696d-dbc3-4112-967a-a5324667ef10 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:29.876392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3506696d-dbc3-4112-967a-a5324667ef10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:30.019407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506696d-dbc3-4112-967a-a5324667ef10 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:30.092952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:30.136099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86783646-8793-4f4d-b000-867027ff8dbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:30.136726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86783646-8793-4f4d-b000-867027ff8dbf service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:30.137017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86783646-8793-4f4d-b000-867027ff8dbf service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:30.137327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86783646-8793-4f4d-b000-867027ff8dbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:30.139206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-86783646-8793-4f4d-b000-867027ff8dbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:30.228869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86783646-8793-4f4d-b000-867027ff8dbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:30.230519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86783646-8793-4f4d-b000-867027ff8dbf service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:30.249318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 720/1416] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:31.164584 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364123c9-f5de-4f6d-a8b7-93a0be64c2a4 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:31.165153 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364123c9-f5de-4f6d-a8b7-93a0be64c2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:31.165153 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364123c9-f5de-4f6d-a8b7-93a0be64c2a4 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:31.166544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:31.172115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc70ff-50b0-4de0-85ba-0e3e1829e5d7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:31.172550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc70ff-50b0-4de0-85ba-0e3e1829e5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:31.172685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc70ff-50b0-4de0-85ba-0e3e1829e5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:31.247964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-83fc70ff-50b0-4de0-85ba-0e3e1829e5d7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:31.249286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-83fc70ff-50b0-4de0-85ba-0e3e1829e5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:31.276044 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:31.346751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:21:31.500448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364123c9-f5de-4f6d-a8b7-93a0be64c2a4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:31.501910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-364123c9-f5de-4f6d-a8b7-93a0be64c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:31.654700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364123c9-f5de-4f6d-a8b7-93a0be64c2a4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:31.695156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:31.770212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23804e-f0d5-4d9d-b122-b6e31a320baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:31.770743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23804e-f0d5-4d9d-b122-b6e31a320baa service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:31.771029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23804e-f0d5-4d9d-b122-b6e31a320baa service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:31.771306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23804e-f0d5-4d9d-b122-b6e31a320baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:31.772955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ef23804e-f0d5-4d9d-b122-b6e31a320baa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:31.871718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23804e-f0d5-4d9d-b122-b6e31a320baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:31.874280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23804e-f0d5-4d9d-b122-b6e31a320baa service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:31.893272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 698/1419] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:32.797723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576dbd-a12d-4a2d-b55c-ffeadee48932 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:32.798297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576dbd-a12d-4a2d-b55c-ffeadee48932 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:32.798297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576dbd-a12d-4a2d-b55c-ffeadee48932 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:32.799658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:21:32.804712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395192-cdd2-4e7f-9f41-4e8d679002ae None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:32.805120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395192-cdd2-4e7f-9f41-4e8d679002ae None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:32.805279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395192-cdd2-4e7f-9f41-4e8d679002ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:32.860791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:32.883137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-26395192-cdd2-4e7f-9f41-4e8d679002ae None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:32.884355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-26395192-cdd2-4e7f-9f41-4e8d679002ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:32.983487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:33.137368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e576dbd-a12d-4a2d-b55c-ffeadee48932 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:33.139529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2e576dbd-a12d-4a2d-b55c-ffeadee48932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:33.288139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e576dbd-a12d-4a2d-b55c-ffeadee48932 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:33.300382 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:33.403507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a4df5-ac49-4a78-bea1-5e7601c2488e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:33.404082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a4df5-ac49-4a78-bea1-5e7601c2488e service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:33.404369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a4df5-ac49-4a78-bea1-5e7601c2488e service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:33.404808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a4df5-ac49-4a78-bea1-5e7601c2488e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:33.406527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-333a4df5-ac49-4a78-bea1-5e7601c2488e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:33.498752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a4df5-ac49-4a78-bea1-5e7601c2488e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:33.500398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a4df5-ac49-4a78-bea1-5e7601c2488e service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:33.519600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 723/1422] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:34.395252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3a939-74fa-4493-b7bb-7bdcd26eca7e None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:34.395821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3a939-74fa-4493-b7bb-7bdcd26eca7e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:34.395821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3a939-74fa-4493-b7bb-7bdcd26eca7e None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:34.397222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:34.402393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028eecc4-f1df-458a-8ddd-74365c3f9bf2 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:34.402879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028eecc4-f1df-458a-8ddd-74365c3f9bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:34.402974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028eecc4-f1df-458a-8ddd-74365c3f9bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:34.423677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:34.482068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-028eecc4-f1df-458a-8ddd-74365c3f9bf2 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:34.484210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-028eecc4-f1df-458a-8ddd-74365c3f9bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:34.582960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:34.734178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3a939-74fa-4493-b7bb-7bdcd26eca7e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:34.735638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-46a3a939-74fa-4493-b7bb-7bdcd26eca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:34.851693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:34.889691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3a939-74fa-4493-b7bb-7bdcd26eca7e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:35.004625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ed0f4-bf88-4843-b35c-26adebee8912 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:35.005249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed0f4-bf88-4843-b35c-26adebee8912 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:35.005562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed0f4-bf88-4843-b35c-26adebee8912 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:35.005836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed0f4-bf88-4843-b35c-26adebee8912 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:35.007473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-655ed0f4-bf88-4843-b35c-26adebee8912 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:35.083585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:35.096972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ed0f4-bf88-4843-b35c-26adebee8912 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:35.098675 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ed0f4-bf88-4843-b35c-26adebee8912 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:35.117812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 701/1425] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:36.028686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026e3c7-b261-41c8-97e1-af19a7874c8c None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:36.029283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026e3c7-b261-41c8-97e1-af19a7874c8c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:36.029483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026e3c7-b261-41c8-97e1-af19a7874c8c None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:36.030795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:36.035644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2d38-1684-47ff-b00f-7cd48baa750d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:36.036102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2d38-1684-47ff-b00f-7cd48baa750d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:36.036208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2d38-1684-47ff-b00f-7cd48baa750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:36.115499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d9bc2d38-1684-47ff-b00f-7cd48baa750d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:36.116734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d9bc2d38-1684-47ff-b00f-7cd48baa750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:36.213560 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:36.367768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026e3c7-b261-41c8-97e1-af19a7874c8c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:36.369321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9026e3c7-b261-41c8-97e1-af19a7874c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:36.454839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:36.517564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026e3c7-b261-41c8-97e1-af19a7874c8c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:36.629199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9db31-4e14-4dd9-9622-8273c6a84c96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:36.629759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9db31-4e14-4dd9-9622-8273c6a84c96 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:36.630045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9db31-4e14-4dd9-9622-8273c6a84c96 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:36.630319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9db31-4e14-4dd9-9622-8273c6a84c96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:36.631954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a7f9db31-4e14-4dd9-9622-8273c6a84c96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:36.679304 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:36.720847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9db31-4e14-4dd9-9622-8273c6a84c96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:36.722477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9db31-4e14-4dd9-9622-8273c6a84c96 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:36.741388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 726/1428] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:21:37.650370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee63e11-ff1f-41d4-a475-03e54be2dbca None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:37.650912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee63e11-ff1f-41d4-a475-03e54be2dbca None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:37.650912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee63e11-ff1f-41d4-a475-03e54be2dbca None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:37.652297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:37.658432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fd1ca-8195-4ba0-8fb7-6f4c2b4f6543 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:37.658906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fd1ca-8195-4ba0-8fb7-6f4c2b4f6543 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:37.659135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fd1ca-8195-4ba0-8fb7-6f4c2b4f6543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:37.738893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-1d8fd1ca-8195-4ba0-8fb7-6f4c2b4f6543 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:37.740213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1d8fd1ca-8195-4ba0-8fb7-6f4c2b4f6543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:37.838592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:37.993265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee63e11-ff1f-41d4-a475-03e54be2dbca None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:37.994951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9ee63e11-ff1f-41d4-a475-03e54be2dbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:38.047413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:38.147394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee63e11-ff1f-41d4-a475-03e54be2dbca None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:38.263061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a42bce-a291-4696-a47a-b83538ac348e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:38.264183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a42bce-a291-4696-a47a-b83538ac348e service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:38.264462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a42bce-a291-4696-a47a-b83538ac348e service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:38.264731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a42bce-a291-4696-a47a-b83538ac348e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:38.266415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-77a42bce-a291-4696-a47a-b83538ac348e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:38.273499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:38.359318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a42bce-a291-4696-a47a-b83538ac348e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:38.360974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a42bce-a291-4696-a47a-b83538ac348e service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:38.380428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 704/1431] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:39.251366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84f2e-a388-4244-a5cb-cf21289056f3 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:39.251900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84f2e-a388-4244-a5cb-cf21289056f3 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:39.251996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84f2e-a388-4244-a5cb-cf21289056f3 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:39.253337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:39.258445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e46d2f-4f03-417f-a5d5-8c58008c0427 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:39.258874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e46d2f-4f03-417f-a5d5-8c58008c0427 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:39.258990 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e46d2f-4f03-417f-a5d5-8c58008c0427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:39.336540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-37e46d2f-4f03-417f-a5d5-8c58008c0427 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:39.337811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-37e46d2f-4f03-417f-a5d5-8c58008c0427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:39.414118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:39.443520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:39.601271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84f2e-a388-4244-a5cb-cf21289056f3 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:39.602818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-33f84f2e-a388-4244-a5cb-cf21289056f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:39.622059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:39.751541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84f2e-a388-4244-a5cb-cf21289056f3 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:39.838176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:39.864523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa159-7b23-41e7-ae0e-d1f07f7b2c48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:39.865071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa159-7b23-41e7-ae0e-d1f07f7b2c48 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:39.865386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa159-7b23-41e7-ae0e-d1f07f7b2c48 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:39.865669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa159-7b23-41e7-ae0e-d1f07f7b2c48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:39.867316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-402aa159-7b23-41e7-ae0e-d1f07f7b2c48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:39.959051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aa159-7b23-41e7-ae0e-d1f07f7b2c48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:39.960659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aa159-7b23-41e7-ae0e-d1f07f7b2c48 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:39.978978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 729/1434] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:40.848536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c141020-ce7f-48a3-a95c-673c75249f4f None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:40.849136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c141020-ce7f-48a3-a95c-673c75249f4f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:40.849136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c141020-ce7f-48a3-a95c-673c75249f4f None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:40.850482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:40.855536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a075e33-b731-4147-a195-73ba2bfdaabc None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:40.856031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a075e33-b731-4147-a195-73ba2bfdaabc None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:40.856142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a075e33-b731-4147-a195-73ba2bfdaabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:40.932889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2a075e33-b731-4147-a195-73ba2bfdaabc None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:40.934078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2a075e33-b731-4147-a195-73ba2bfdaabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:40.977123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:41.032369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:41.181568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c141020-ce7f-48a3-a95c-673c75249f4f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:41.183299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4c141020-ce7f-48a3-a95c-673c75249f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:41.332995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c141020-ce7f-48a3-a95c-673c75249f4f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:41.390954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:41.450484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd7d2e-1d64-40b6-87fc-d345885dfcf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:41.451121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd7d2e-1d64-40b6-87fc-d345885dfcf7 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:41.451412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd7d2e-1d64-40b6-87fc-d345885dfcf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:41.451684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd7d2e-1d64-40b6-87fc-d345885dfcf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:41.453378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-60fd7d2e-1d64-40b6-87fc-d345885dfcf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:41.545541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd7d2e-1d64-40b6-87fc-d345885dfcf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:41.547171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd7d2e-1d64-40b6-87fc-d345885dfcf7 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:41.565645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 707/1437] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:42.455239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136721b-241f-4bab-9cf9-0c90b92a9dda None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:42.455784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136721b-241f-4bab-9cf9-0c90b92a9dda None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:42.455878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136721b-241f-4bab-9cf9-0c90b92a9dda None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:42.457198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:42.462454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d49672-4baa-44e3-a6e3-136e13a25249 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:42.462872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d49672-4baa-44e3-a6e3-136e13a25249 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:42.463056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d49672-4baa-44e3-a6e3-136e13a25249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:42.553359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a0d49672-4baa-44e3-a6e3-136e13a25249 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:42.554592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a0d49672-4baa-44e3-a6e3-136e13a25249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:42.556691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:42.652581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:42.805582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136721b-241f-4bab-9cf9-0c90b92a9dda None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:42.807245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7136721b-241f-4bab-9cf9-0c90b92a9dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:42.955309 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136721b-241f-4bab-9cf9-0c90b92a9dda None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:42.982857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:43.070011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1509ef-96cb-45ac-95e0-811b1dcff298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:43.070671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1509ef-96cb-45ac-95e0-811b1dcff298 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:43.070960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1509ef-96cb-45ac-95e0-811b1dcff298 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:43.071244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1509ef-96cb-45ac-95e0-811b1dcff298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:43.072917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7c1509ef-96cb-45ac-95e0-811b1dcff298 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:43.164351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1509ef-96cb-45ac-95e0-811b1dcff298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:43.165998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1509ef-96cb-45ac-95e0-811b1dcff298 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:43.184406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 732/1440] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:45.084953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b784f31-9121-44b5-a69e-0025e2b54595 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:45.085519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b784f31-9121-44b5-a69e-0025e2b54595 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:45.085519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b784f31-9121-44b5-a69e-0025e2b54595 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:45.086974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:45.092504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44c3d3-74e2-460d-85b8-f57a8d54af3d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:45.092948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44c3d3-74e2-460d-85b8-f57a8d54af3d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:45.093063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44c3d3-74e2-460d-85b8-f57a8d54af3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:45.257627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:45.282864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-9a44c3d3-74e2-460d-85b8-f57a8d54af3d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:45.284185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9a44c3d3-74e2-460d-85b8-f57a8d54af3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:45.381389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:45.420839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b784f31-9121-44b5-a69e-0025e2b54595 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:45.422417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9b784f31-9121-44b5-a69e-0025e2b54595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:45.526419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b784f31-9121-44b5-a69e-0025e2b54595 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:45.559523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:45.603470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005bad1-a555-4aec-b029-1a4642d0c2fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:45.604061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005bad1-a555-4aec-b029-1a4642d0c2fd service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:45.604342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005bad1-a555-4aec-b029-1a4642d0c2fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:45.604615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005bad1-a555-4aec-b029-1a4642d0c2fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:45.606263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0005bad1-a555-4aec-b029-1a4642d0c2fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:45.719028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005bad1-a555-4aec-b029-1a4642d0c2fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:45.720488 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005bad1-a555-4aec-b029-1a4642d0c2fd service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:45.739861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 710/1443] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:21:47.932364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be941a-3678-4c3e-8576-4396d1f1dc38 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:47.933012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be941a-3678-4c3e-8576-4396d1f1dc38 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:47.933012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be941a-3678-4c3e-8576-4396d1f1dc38 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:47.934458 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:47.940616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ac241-08f9-46d6-bcee-cf474115efcd None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:47.941058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ac241-08f9-46d6-bcee-cf474115efcd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:47.941256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ac241-08f9-46d6-bcee-cf474115efcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:48.007723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c88ac241-08f9-46d6-bcee-cf474115efcd None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:48.009012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c88ac241-08f9-46d6-bcee-cf474115efcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:48.022344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:48.109021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:48.147845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be941a-3678-4c3e-8576-4396d1f1dc38 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:48.149481 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-33be941a-3678-4c3e-8576-4396d1f1dc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:48.254408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be941a-3678-4c3e-8576-4396d1f1dc38 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:48.267040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:48.329757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b0f87-18c7-4cd5-bb7a-5f8d1de4a166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:48.330384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b0f87-18c7-4cd5-bb7a-5f8d1de4a166 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:48.330678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b0f87-18c7-4cd5-bb7a-5f8d1de4a166 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:48.330955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b0f87-18c7-4cd5-bb7a-5f8d1de4a166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:48.332659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-052b0f87-18c7-4cd5-bb7a-5f8d1de4a166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:48.443770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b0f87-18c7-4cd5-bb7a-5f8d1de4a166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:48.445213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b0f87-18c7-4cd5-bb7a-5f8d1de4a166 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:48.463425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 735/1446] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:50.854374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcffce-2b26-4ea3-ae7b-0ce27db694e0 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:50.855010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcffce-2b26-4ea3-ae7b-0ce27db694e0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:50.855010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcffce-2b26-4ea3-ae7b-0ce27db694e0 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:50.856538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:50.861924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700e74b-a40e-46e3-b099-81c5e33a6481 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:50.862422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700e74b-a40e-46e3-b099-81c5e33a6481 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:50.862539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700e74b-a40e-46e3-b099-81c5e33a6481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:50.911617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:50.929689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f700e74b-a40e-46e3-b099-81c5e33a6481 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:50.944687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f700e74b-a40e-46e3-b099-81c5e33a6481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:51.033788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:51.073820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcffce-2b26-4ea3-ae7b-0ce27db694e0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:51.075469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f8dcffce-2b26-4ea3-ae7b-0ce27db694e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:51.183897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcffce-2b26-4ea3-ae7b-0ce27db694e0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:51.224011 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:51.258616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09b1bf-837b-4b5f-b05a-d55109318078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:51.259241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09b1bf-837b-4b5f-b05a-d55109318078 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:51.259524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09b1bf-837b-4b5f-b05a-d55109318078 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:51.259802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09b1bf-837b-4b5f-b05a-d55109318078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:51.262169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fe09b1bf-837b-4b5f-b05a-d55109318078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:51.498899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09b1bf-837b-4b5f-b05a-d55109318078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:51.500360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09b1bf-837b-4b5f-b05a-d55109318078 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:51.519965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 713/1449] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:53.686872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efebe0f-3c1a-40ed-9dad-ff071e68dceb None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:53.687414 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efebe0f-3c1a-40ed-9dad-ff071e68dceb None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:53.687414 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efebe0f-3c1a-40ed-9dad-ff071e68dceb None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:53.688712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:53.694087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fa8d4-dfbf-4b0c-8c8a-768db169de46 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:53.694553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fa8d4-dfbf-4b0c-8c8a-768db169de46 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:53.694651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fa8d4-dfbf-4b0c-8c8a-768db169de46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:53.760732 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-394fa8d4-dfbf-4b0c-8c8a-768db169de46 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:53.761976 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-394fa8d4-dfbf-4b0c-8c8a-768db169de46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:53.783363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:53.860654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:53.899618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efebe0f-3c1a-40ed-9dad-ff071e68dceb None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:53.901195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0efebe0f-3c1a-40ed-9dad-ff071e68dceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:54.010262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efebe0f-3c1a-40ed-9dad-ff071e68dceb None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:54.032635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:54.089195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0d224-792b-4644-a9cb-f83631acdeb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:54.089848 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0d224-792b-4644-a9cb-f83631acdeb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:54.090123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0d224-792b-4644-a9cb-f83631acdeb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:54.090396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0d224-792b-4644-a9cb-f83631acdeb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:54.092736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2fd0d224-792b-4644-a9cb-f83631acdeb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:54.205460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0d224-792b-4644-a9cb-f83631acdeb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:54.206913 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0d224-792b-4644-a9cb-f83631acdeb0 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:54.225541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 738/1452] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:56.311707 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d04f0f-b490-4d26-9b67-66b1686ff4f7 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:56.312332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d04f0f-b490-4d26-9b67-66b1686ff4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:56.312332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d04f0f-b490-4d26-9b67-66b1686ff4f7 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:56.313646 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:56.319282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ad8ed-e1c1-43ed-90fb-fcec29b09885 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:56.319824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ad8ed-e1c1-43ed-90fb-fcec29b09885 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:56.319824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ad8ed-e1c1-43ed-90fb-fcec29b09885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:56.389279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:56.392408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-c50ad8ed-e1c1-43ed-90fb-fcec29b09885 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:56.393722 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c50ad8ed-e1c1-43ed-90fb-fcec29b09885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:56.491844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:56.530895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d04f0f-b490-4d26-9b67-66b1686ff4f7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:56.532611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d8d04f0f-b490-4d26-9b67-66b1686ff4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:56.638537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d04f0f-b490-4d26-9b67-66b1686ff4f7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:56.688317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:56.712098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934d354-0e5b-4c91-84bf-4bf92dcbad80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:56.712730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934d354-0e5b-4c91-84bf-4bf92dcbad80 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:56.713048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934d354-0e5b-4c91-84bf-4bf92dcbad80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:56.713353 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934d354-0e5b-4c91-84bf-4bf92dcbad80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:56.715260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6934d354-0e5b-4c91-84bf-4bf92dcbad80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:56.829815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934d354-0e5b-4c91-84bf-4bf92dcbad80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:56.831265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934d354-0e5b-4c91-84bf-4bf92dcbad80 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:56.851763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 716/1455] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:21:58.938204 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c59e9-be65-4850-b3af-cfe025bda977 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:58.938782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c59e9-be65-4850-b3af-cfe025bda977 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:58.938782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c59e9-be65-4850-b3af-cfe025bda977 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:58.940128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:21:58.945533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c7a5c-fe3d-496d-9bcc-6d084a9c1f21 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:58.945979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c7a5c-fe3d-496d-9bcc-6d084a9c1f21 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:58.946102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c7a5c-fe3d-496d-9bcc-6d084a9c1f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:59.013451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8e8c7a5c-fe3d-496d-9bcc-6d084a9c1f21 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:21:59.014674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8e8c7a5c-fe3d-496d-9bcc-6d084a9c1f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:59.043684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:59.113991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:21:59.153378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c59e9-be65-4850-b3af-cfe025bda977 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:21:59.154916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3a5c59e9-be65-4850-b3af-cfe025bda977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:59.261992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c59e9-be65-4850-b3af-cfe025bda977 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:21:59.290288 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:21:59.335713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532c063-28cd-4581-9bba-4982564c2188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:21:59.336337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532c063-28cd-4581-9bba-4982564c2188 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:21:59.336616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532c063-28cd-4581-9bba-4982564c2188 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:21:59.336886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532c063-28cd-4581-9bba-4982564c2188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:21:59.338555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9532c063-28cd-4581-9bba-4982564c2188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:21:59.451079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532c063-28cd-4581-9bba-4982564c2188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:21:59.452554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532c063-28cd-4581-9bba-4982564c2188 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:21:59.470645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 741/1458] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:01.561334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f38933-ec71-42c0-9135-08cbd397408c None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:01.561873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f38933-ec71-42c0-9135-08cbd397408c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:01.561998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f38933-ec71-42c0-9135-08cbd397408c None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:01.563389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:01.568931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79851dbb-25a5-4848-a304-e3cc4e1b6a1c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:01.569422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79851dbb-25a5-4848-a304-e3cc4e1b6a1c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:01.569593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79851dbb-25a5-4848-a304-e3cc4e1b6a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:01.636426 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-79851dbb-25a5-4848-a304-e3cc4e1b6a1c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:01.637590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-79851dbb-25a5-4848-a304-e3cc4e1b6a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:01.646506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:01.737129 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:01.775351 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f38933-ec71-42c0-9135-08cbd397408c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:01.776890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-23f38933-ec71-42c0-9135-08cbd397408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:01.883636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f38933-ec71-42c0-9135-08cbd397408c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:01.940412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:01.959857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a91f24-2cb2-4dc1-980a-ef320c99f404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:01.960487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a91f24-2cb2-4dc1-980a-ef320c99f404 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:01.960777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a91f24-2cb2-4dc1-980a-ef320c99f404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:01.961072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a91f24-2cb2-4dc1-980a-ef320c99f404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:01.962817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-08a91f24-2cb2-4dc1-980a-ef320c99f404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:02.076514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a91f24-2cb2-4dc1-980a-ef320c99f404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:02.078012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a91f24-2cb2-4dc1-980a-ef320c99f404 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:02.096480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 719/1461] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:04.176876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4619e-6b39-467d-a8af-dd89ca7e8273 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:04.177478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4619e-6b39-467d-a8af-dd89ca7e8273 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:04.177478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4619e-6b39-467d-a8af-dd89ca7e8273 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:04.178830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:04.184225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55263c-fefb-4378-9418-d5cfe764a215 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:04.184649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55263c-fefb-4378-9418-d5cfe764a215 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:04.184977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55263c-fefb-4378-9418-d5cfe764a215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:04.252765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ac55263c-fefb-4378-9418-d5cfe764a215 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:04.254019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ac55263c-fefb-4378-9418-d5cfe764a215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:04.294178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:04.353907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:04.392855 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4619e-6b39-467d-a8af-dd89ca7e8273 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:04.394807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-88e4619e-6b39-467d-a8af-dd89ca7e8273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:04.510999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4619e-6b39-467d-a8af-dd89ca7e8273 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:04.549964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:04.587950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32a8a5-4c3b-4689-93b6-83330fc3f56b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:04.588583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32a8a5-4c3b-4689-93b6-83330fc3f56b service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:04.588875 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32a8a5-4c3b-4689-93b6-83330fc3f56b service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:04.589157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32a8a5-4c3b-4689-93b6-83330fc3f56b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:04.590851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2b32a8a5-4c3b-4689-93b6-83330fc3f56b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:04.700517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32a8a5-4c3b-4689-93b6-83330fc3f56b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:04.702008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32a8a5-4c3b-4689-93b6-83330fc3f56b service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:04.720531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 744/1464] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:06.786977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3fbe2-1f47-4a86-ad70-b3bcd1b0f031 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:06.787554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3fbe2-1f47-4a86-ad70-b3bcd1b0f031 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:06.787554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3fbe2-1f47-4a86-ad70-b3bcd1b0f031 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:06.788921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:06.794796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0ee54-e590-4e43-ae4a-6b3efeaca66d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:06.795323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0ee54-e590-4e43-ae4a-6b3efeaca66d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:06.795323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0ee54-e590-4e43-ae4a-6b3efeaca66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:06.860077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-a2c0ee54-e590-4e43-ae4a-6b3efeaca66d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:06.861288 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a2c0ee54-e590-4e43-ae4a-6b3efeaca66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:06.880117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:06.959761 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:06.997557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3fbe2-1f47-4a86-ad70-b3bcd1b0f031 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:06.999217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6ba3fbe2-1f47-4a86-ad70-b3bcd1b0f031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:07.103993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3fbe2-1f47-4a86-ad70-b3bcd1b0f031 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:07.171368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:07.177968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52537f-615c-472a-addd-004f412d0235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:07.178594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52537f-615c-472a-addd-004f412d0235 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:07.183009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52537f-615c-472a-addd-004f412d0235 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:07.183009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52537f-615c-472a-addd-004f412d0235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:07.183009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8f52537f-615c-472a-addd-004f412d0235 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:07.299144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52537f-615c-472a-addd-004f412d0235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:07.300760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52537f-615c-472a-addd-004f412d0235 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:07.319656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 722/1467] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:22:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:09.475770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104de1a2-2119-4f0d-9d71-c344c699afbe None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:09.476330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104de1a2-2119-4f0d-9d71-c344c699afbe None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:09.476431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104de1a2-2119-4f0d-9d71-c344c699afbe None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:09.477919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:09.484243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505a35d-65cc-4747-878f-5bd26463e67d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:09.484684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505a35d-65cc-4747-878f-5bd26463e67d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:09.484814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505a35d-65cc-4747-878f-5bd26463e67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:09.557140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c505a35d-65cc-4747-878f-5bd26463e67d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:09.558402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c505a35d-65cc-4747-878f-5bd26463e67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:09.609058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:09.659005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:09.695577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104de1a2-2119-4f0d-9d71-c344c699afbe None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:09.697131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-104de1a2-2119-4f0d-9d71-c344c699afbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:09.806807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104de1a2-2119-4f0d-9d71-c344c699afbe None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:09.855110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:09.884547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3379df-ea8d-46c5-a5f1-7b67822096e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:09.885169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3379df-ea8d-46c5-a5f1-7b67822096e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:09.885499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3379df-ea8d-46c5-a5f1-7b67822096e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:09.885784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3379df-ea8d-46c5-a5f1-7b67822096e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:09.887438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8f3379df-ea8d-46c5-a5f1-7b67822096e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:09.996708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3379df-ea8d-46c5-a5f1-7b67822096e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:09.998707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3379df-ea8d-46c5-a5f1-7b67822096e9 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:10.017863 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 747/1470] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:22:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:19.657137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=22 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:20.015513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=66 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:29.933830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62809c93-f004-47cf-8e20-9c062ec62c42 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:29.934562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62809c93-f004-47cf-8e20-9c062ec62c42 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:29.934562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62809c93-f004-47cf-8e20-9c062ec62c42 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:29.935746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 724/1471] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:29.941043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bbcc9-ced8-4d13-991f-24943129f6da None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:29.941573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bbcc9-ced8-4d13-991f-24943129f6da None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:29.941679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bbcc9-ced8-4d13-991f-24943129f6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:30.008263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-353bbcc9-ced8-4d13-991f-24943129f6da None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:30.009539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-353bbcc9-ced8-4d13-991f-24943129f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:30.108280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 748/1472] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:30.193191 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14948dc-a769-4df8-9c1f-22351337aedd None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:30.193513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14948dc-a769-4df8-9c1f-22351337aedd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:30.193793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14948dc-a769-4df8-9c1f-22351337aedd None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:30.195140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 725/1473] 10.222.0.69 () {54 vars in 966 bytes} [Thu Jul 28 06:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:30.200551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c79a2-6956-4285-bf80-9802cbad71bd None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:30.200847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c79a2-6956-4285-bf80-9802cbad71bd None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:30.201134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c79a2-6956-4285-bf80-9802cbad71bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:30.278897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-105c79a2-6956-4285-bf80-9802cbad71bd None None] MFA Rules not processed for user `cc4fe04b11ec412e8f4eb67c6e54b8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:30.280118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-105c79a2-6956-4285-bf80-9802cbad71bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:30.300942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:30.359618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 749/1474] 10.222.0.69 () {58 vars in 1063 bytes} [Thu Jul 28 06:22:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:30.367501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14948dc-a769-4df8-9c1f-22351337aedd None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:30.369007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e14948dc-a769-4df8-9c1f-22351337aedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:30.472377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14948dc-a769-4df8-9c1f-22351337aedd None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:30.544941 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e5874-19de-48b0-ae49-11809328c965 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4fe04b11ec412e8f4eb67c6e54b8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:30.545592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e5874-19de-48b0-ae49-11809328c965 service glance] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:30.545872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e5874-19de-48b0-ae49-11809328c965 service glance] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:30.546145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e5874-19de-48b0-ae49-11809328c965 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:30.547767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-be5e5874-19de-48b0-ae49-11809328c965 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:30.600601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:30.656144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e5874-19de-48b0-ae49-11809328c965 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:30.657660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e5874-19de-48b0-ae49-11809328c965 service glance] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:30.676255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 726/1475] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:22:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:32.773365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc17f5e-4c61-4c96-98d8-3256b880ee7b None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:32.773952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc17f5e-4c61-4c96-98d8-3256b880ee7b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:32.773952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc17f5e-4c61-4c96-98d8-3256b880ee7b None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:32.775326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 750/1476] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:32.780938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88e77e-4f5b-4c24-86ca-49af9d2ba200 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:32.781435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88e77e-4f5b-4c24-86ca-49af9d2ba200 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:32.781541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88e77e-4f5b-4c24-86ca-49af9d2ba200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:32.846428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0f88e77e-4f5b-4c24-86ca-49af9d2ba200 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:32.847632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0f88e77e-4f5b-4c24-86ca-49af9d2ba200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:32.944068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 727/1477] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:32.968044 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc17f5e-4c61-4c96-98d8-3256b880ee7b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:32.969619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3bc17f5e-4c61-4c96-98d8-3256b880ee7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:33.093502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc17f5e-4c61-4c96-98d8-3256b880ee7b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:33.141600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:33.186794 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f0628-2f35-40be-8751-3a3f99ced976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:33.187415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f0628-2f35-40be-8751-3a3f99ced976 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:33.187703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f0628-2f35-40be-8751-3a3f99ced976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:33.187980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f0628-2f35-40be-8751-3a3f99ced976 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:33.188679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f0628-2f35-40be-8751-3a3f99ced976 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:33.190443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WARNING py.warnings [None req-701f0628-2f35-40be-8751-3a3f99ced976 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:22:33.190443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: warnings.warn(msg) Jul 28 06:22:33.190443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: Jul 28 06:22:33.191592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f0628-2f35-40be-8751-3a3f99ced976 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:33.210826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 751/1478] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 28 06:22:32 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:22:35.267526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4148a1-55a1-4a40-af47-2da5c8d69863 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:35.268062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4148a1-55a1-4a40-af47-2da5c8d69863 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:35.268062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4148a1-55a1-4a40-af47-2da5c8d69863 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:35.269413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 728/1479] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:35.274860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996da265-8756-4126-9a5a-589909ad885d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:35.275335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996da265-8756-4126-9a5a-589909ad885d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:35.275453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996da265-8756-4126-9a5a-589909ad885d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:35.344124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-996da265-8756-4126-9a5a-589909ad885d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:35.345432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-996da265-8756-4126-9a5a-589909ad885d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:35.452330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 752/1480] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:35.496743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4148a1-55a1-4a40-af47-2da5c8d69863 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:35.498342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7e4148a1-55a1-4a40-af47-2da5c8d69863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:35.508330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:35.651544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4148a1-55a1-4a40-af47-2da5c8d69863 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:35.739202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:35.772013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898ff26-5763-4e22-87db-bc071434d8bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:35.772641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898ff26-5763-4e22-87db-bc071434d8bb service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:35.772927 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898ff26-5763-4e22-87db-bc071434d8bb service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:35.773202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898ff26-5763-4e22-87db-bc071434d8bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:35.775005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6898ff26-5763-4e22-87db-bc071434d8bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:35.868777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898ff26-5763-4e22-87db-bc071434d8bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:35.870433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898ff26-5763-4e22-87db-bc071434d8bb service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:35.890877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 729/1481] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:22:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:38.125549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d127a9-adcb-4295-8c64-65730606f5f6 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:38.126096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d127a9-adcb-4295-8c64-65730606f5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:38.126096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d127a9-adcb-4295-8c64-65730606f5f6 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:38.127541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 753/1482] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:38.132981 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70271170-46d1-4069-acad-7dab16bf494b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:38.133787 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70271170-46d1-4069-acad-7dab16bf494b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:38.134070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70271170-46d1-4069-acad-7dab16bf494b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:38.202595 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-70271170-46d1-4069-acad-7dab16bf494b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:38.203819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-70271170-46d1-4069-acad-7dab16bf494b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:38.250835 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:38.302876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 730/1483] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:38.340248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d127a9-adcb-4295-8c64-65730606f5f6 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:38.341932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-95d127a9-adcb-4295-8c64-65730606f5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:38.367530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:38.448366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d127a9-adcb-4295-8c64-65730606f5f6 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:38.524441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f554c4-bd1c-4eb0-b885-7ad6346d69c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:38.525072 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f554c4-bd1c-4eb0-b885-7ad6346d69c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:38.525391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f554c4-bd1c-4eb0-b885-7ad6346d69c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:38.525677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f554c4-bd1c-4eb0-b885-7ad6346d69c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:38.528127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-17f554c4-bd1c-4eb0-b885-7ad6346d69c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:38.605420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:38.641346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f554c4-bd1c-4eb0-b885-7ad6346d69c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:38.642734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f554c4-bd1c-4eb0-b885-7ad6346d69c1 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:38.661919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 754/1484] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:22:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:40.804517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692f731-e5cf-493e-8194-9155450a33a1 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:40.805075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692f731-e5cf-493e-8194-9155450a33a1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:40.805075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692f731-e5cf-493e-8194-9155450a33a1 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:40.806531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 731/1485] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:40.812580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2013f7-fddb-48ec-88a9-4b1339cd1ce6 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:40.813127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2013f7-fddb-48ec-88a9-4b1339cd1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:40.813226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2013f7-fddb-48ec-88a9-4b1339cd1ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:40.880949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2a2013f7-fddb-48ec-88a9-4b1339cd1ce6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:40.882179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2a2013f7-fddb-48ec-88a9-4b1339cd1ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:40.977915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 755/1486] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:41.016968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692f731-e5cf-493e-8194-9155450a33a1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:41.018624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2692f731-e5cf-493e-8194-9155450a33a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:41.128908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692f731-e5cf-493e-8194-9155450a33a1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:41.202502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:41.204286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc93ca9-1ad6-46d2-aece-557f3fa557bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:41.204905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc93ca9-1ad6-46d2-aece-557f3fa557bc service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:41.205188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc93ca9-1ad6-46d2-aece-557f3fa557bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:41.205493 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc93ca9-1ad6-46d2-aece-557f3fa557bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:41.207126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7fc93ca9-1ad6-46d2-aece-557f3fa557bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:41.317660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc93ca9-1ad6-46d2-aece-557f3fa557bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:41.319109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc93ca9-1ad6-46d2-aece-557f3fa557bc service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:41.337753 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 732/1487] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:22:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:43.451885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e59a7-c4b7-4d5f-b69f-5d3265b60397 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:43.452445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e59a7-c4b7-4d5f-b69f-5d3265b60397 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:43.452445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e59a7-c4b7-4d5f-b69f-5d3265b60397 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:43.453850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 756/1488] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:43.459032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6c4ac-1905-4879-9a01-6f49ae0b13d9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:43.459444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6c4ac-1905-4879-9a01-6f49ae0b13d9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:43.459601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6c4ac-1905-4879-9a01-6f49ae0b13d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:43.527911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-22a6c4ac-1905-4879-9a01-6f49ae0b13d9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:43.528376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-22a6c4ac-1905-4879-9a01-6f49ae0b13d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:43.580992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:43.626344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 733/1489] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:43.666457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e59a7-c4b7-4d5f-b69f-5d3265b60397 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:43.668706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d43e59a7-c4b7-4d5f-b69f-5d3265b60397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:43.706521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:43.776251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e59a7-c4b7-4d5f-b69f-5d3265b60397 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:43.849416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be321c-c160-498c-8bd0-a43bbd1dd6dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:43.850040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be321c-c160-498c-8bd0-a43bbd1dd6dc service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:43.850328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be321c-c160-498c-8bd0-a43bbd1dd6dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:43.850611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be321c-c160-498c-8bd0-a43bbd1dd6dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:43.852255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b0be321c-c160-498c-8bd0-a43bbd1dd6dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:43.939378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:43.969923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be321c-c160-498c-8bd0-a43bbd1dd6dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:43.971410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be321c-c160-498c-8bd0-a43bbd1dd6dc service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:43.990593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 757/1490] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:22:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:46.445283 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1a67b-f99a-4972-893d-d586c1269927 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:46.445956 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1a67b-f99a-4972-893d-d586c1269927 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:46.445956 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1a67b-f99a-4972-893d-d586c1269927 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:46.447266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 734/1491] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 28 06:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:46.453579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7b099-a5ab-412d-b61d-9a970a83d440 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:46.454095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7b099-a5ab-412d-b61d-9a970a83d440 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:46.454189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7b099-a5ab-412d-b61d-9a970a83d440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:46.526873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-69d7b099-a5ab-412d-b61d-9a970a83d440 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:46.528278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-69d7b099-a5ab-412d-b61d-9a970a83d440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:46.631132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 758/1492] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:22:46.670691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1a67b-f99a-4972-893d-d586c1269927 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:46.672278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-85e1a67b-f99a-4972-893d-d586c1269927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:46.778911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1a67b-f99a-4972-893d-d586c1269927 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:46.854263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd32893-4485-4d29-9bb6-30054e01d721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:46.855032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd32893-4485-4d29-9bb6-30054e01d721 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:46.855304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd32893-4485-4d29-9bb6-30054e01d721 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:46.855585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd32893-4485-4d29-9bb6-30054e01d721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:46.857266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bfd32893-4485-4d29-9bb6-30054e01d721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:46.867615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:46.970540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd32893-4485-4d29-9bb6-30054e01d721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:46.971927 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd32893-4485-4d29-9bb6-30054e01d721 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:46.991852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 735/1493] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:22:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:22:56.628795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=82 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:56.989006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=50 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:24.143865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7e24a-ad41-4a0d-90db-6b992eaf1c39 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:24.144473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7e24a-ad41-4a0d-90db-6b992eaf1c39 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:24.144473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7e24a-ad41-4a0d-90db-6b992eaf1c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:24.226830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-3af7e24a-ad41-4a0d-90db-6b992eaf1c39 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:24.228065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3af7e24a-ad41-4a0d-90db-6b992eaf1c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:24.328097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 759/1494] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:24:24.336933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e8617-0e6d-4f97-9ead-dc57b07b7ae7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:24.337529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e8617-0e6d-4f97-9ead-dc57b07b7ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:24.337697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e8617-0e6d-4f97-9ead-dc57b07b7ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:24.422998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-cd8e8617-0e6d-4f97-9ead-dc57b07b7ae7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:24.424141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cd8e8617-0e6d-4f97-9ead-dc57b07b7ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:24.525746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 736/1495] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:24:24.530477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7e24a-ad41-4a0d-90db-6b992eaf1c39 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:24.531728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3af7e24a-ad41-4a0d-90db-6b992eaf1c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:24.544345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7e24a-ad41-4a0d-90db-6b992eaf1c39 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:24.614352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:24.648902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:24.649554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:24.649844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:24.650127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:24.650828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:24.651246 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:24.652368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WARNING py.warnings [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:24:24.652368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: warnings.warn(msg) Jul 28 06:24:24.652368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: Jul 28 06:24:24.653393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:24.665807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 760/1496] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:24.674177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e8617-0e6d-4f97-9ead-dc57b07b7ae7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:24.684122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e8617-0e6d-4f97-9ead-dc57b07b7ae7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:24.750518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:24.786480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba533f8-f444-4ffc-a916-9761f881add2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:24.787160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba533f8-f444-4ffc-a916-9761f881add2 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:24.787462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba533f8-f444-4ffc-a916-9761f881add2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:24.787763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba533f8-f444-4ffc-a916-9761f881add2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:24.788496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba533f8-f444-4ffc-a916-9761f881add2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:24.790266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-6ba533f8-f444-4ffc-a916-9761f881add2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:24:24.790266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Jul 28 06:24:24.790266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: Jul 28 06:24:24.791530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba533f8-f444-4ffc-a916-9761f881add2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.161119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 737/1497] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:24:24 2022] POST /identity/v3/projects => generated 377 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:25.165395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.166991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:25.201407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6b02-0cc5-4a66-801c-0fb54d063072 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.294145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651012a-9395-4d63-accd-bb50878b0444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.294781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651012a-9395-4d63-accd-bb50878b0444 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.295073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651012a-9395-4d63-accd-bb50878b0444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.295363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651012a-9395-4d63-accd-bb50878b0444 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.296031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651012a-9395-4d63-accd-bb50878b0444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:25.297058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WARNING py.warnings [None req-1651012a-9395-4d63-accd-bb50878b0444 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:24:25.297058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: warnings.warn(msg) Jul 28 06:24:25.297058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: Jul 28 06:24:25.298316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651012a-9395-4d63-accd-bb50878b0444 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.398604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 761/1498] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:24:25 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:25.402816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba533f8-f444-4ffc-a916-9761f881add2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.404445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6ba533f8-f444-4ffc-a916-9761f881add2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:25.471481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba533f8-f444-4ffc-a916-9761f881add2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.557216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:25.567652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0255c67-4b72-4fe8-b92b-6f1ea8043d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.568284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0255c67-4b72-4fe8-b92b-6f1ea8043d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.568573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0255c67-4b72-4fe8-b92b-6f1ea8043d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.568982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0255c67-4b72-4fe8-b92b-6f1ea8043d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.569727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0255c67-4b72-4fe8-b92b-6f1ea8043d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:25.570776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-e0255c67-4b72-4fe8-b92b-6f1ea8043d53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:24:25.570776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Jul 28 06:24:25.570776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: Jul 28 06:24:25.571680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0255c67-4b72-4fe8-b92b-6f1ea8043d53 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.587515 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 738/1499] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:24:25.589537 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651012a-9395-4d63-accd-bb50878b0444 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.590381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1651012a-9395-4d63-accd-bb50878b0444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:25.602504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651012a-9395-4d63-accd-bb50878b0444 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.628699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:25.699763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd1b25-08fe-43e9-aacb-891bbcb0a39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.700386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd1b25-08fe-43e9-aacb-891bbcb0a39a admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.700712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd1b25-08fe-43e9-aacb-891bbcb0a39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.700993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd1b25-08fe-43e9-aacb-891bbcb0a39a admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.701727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd1b25-08fe-43e9-aacb-891bbcb0a39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:25.702755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WARNING py.warnings [None req-a4cd1b25-08fe-43e9-aacb-891bbcb0a39a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:24:25.702755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: warnings.warn(msg) Jul 28 06:24:25.702755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: Jul 28 06:24:25.703647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd1b25-08fe-43e9-aacb-891bbcb0a39a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.717416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 762/1500] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:24:25.721808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0255c67-4b72-4fe8-b92b-6f1ea8043d53 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.723372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e0255c67-4b72-4fe8-b92b-6f1ea8043d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:25.765613 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0255c67-4b72-4fe8-b92b-6f1ea8043d53 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.863821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fa469-f465-4fbc-9352-f01ebfd83e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.864500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fa469-f465-4fbc-9352-f01ebfd83e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.864790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fa469-f465-4fbc-9352-f01ebfd83e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.865604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fa469-f465-4fbc-9352-f01ebfd83e5c admin admin] PATH_INFO: `/v3/projects/9fbd04a98eb04e07a901d3bfd7047453/users/829d523656084504b754a0a3bf2d5920/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.888899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fa469-f465-4fbc-9352-f01ebfd83e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fbd04a98eb04e07a901d3bfd7047453, user_id=829d523656084504b754a0a3bf2d5920, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:25.889900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-a13fa469-f465-4fbc-9352-f01ebfd83e5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:24:25.889900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Jul 28 06:24:25.889900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: Jul 28 06:24:25.891866 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fa469-f465-4fbc-9352-f01ebfd83e5c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.963201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 739/1501] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:24:25 2022] PUT /identity/v3/projects/9fbd04a98eb04e07a901d3bfd7047453/users/829d523656084504b754a0a3bf2d5920/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:24:25.975715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c5de6-32bd-4830-bf7c-fd44f39f1171 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.976008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c5de6-32bd-4830-bf7c-fd44f39f1171 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.976295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c5de6-32bd-4830-bf7c-fd44f39f1171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.036720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2b6c5de6-32bd-4830-bf7c-fd44f39f1171 None None] MFA Rules not processed for user `829d523656084504b754a0a3bf2d5920`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:26.038042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2b6c5de6-32bd-4830-bf7c-fd44f39f1171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:26.122785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 763/1502] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:24:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:24:26.136430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52348c40-3c5b-4acc-82f2-cf6271bc6756 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.136822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52348c40-3c5b-4acc-82f2-cf6271bc6756 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.137143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52348c40-3c5b-4acc-82f2-cf6271bc6756 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.138541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 740/1503] 10.222.0.69 () {54 vars in 964 bytes} [Thu Jul 28 06:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:26.143766 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d11ac-2b44-4a37-8d73-f9b3b40057c1 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.144079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d11ac-2b44-4a37-8d73-f9b3b40057c1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.144372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d11ac-2b44-4a37-8d73-f9b3b40057c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.176169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:26.221666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-0a7d11ac-2b44-4a37-8d73-f9b3b40057c1 None None] MFA Rules not processed for user `cbbbf34c238d44db8588d56f35827390`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:26.222961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0a7d11ac-2b44-4a37-8d73-f9b3b40057c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:26.298442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 764/1504] 10.222.0.69 () {58 vars in 1061 bytes} [Thu Jul 28 06:24:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:24:26.306744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52348c40-3c5b-4acc-82f2-cf6271bc6756 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.308288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-52348c40-3c5b-4acc-82f2-cf6271bc6756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:26.360360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:26.419579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52348c40-3c5b-4acc-82f2-cf6271bc6756 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.495039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823d03b-a7cc-4a9f-81c4-e46247740f58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.495669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823d03b-a7cc-4a9f-81c4-e46247740f58 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.495957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823d03b-a7cc-4a9f-81c4-e46247740f58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.496238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823d03b-a7cc-4a9f-81c4-e46247740f58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.497914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3823d03b-a7cc-4a9f-81c4-e46247740f58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:26.602621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823d03b-a7cc-4a9f-81c4-e46247740f58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:26.604075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823d03b-a7cc-4a9f-81c4-e46247740f58 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:26.607912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:26.622106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 741/1505] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:24:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:24:26.667874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08166b9e-98d7-479d-8778-cd27227d6646 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.668186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08166b9e-98d7-479d-8778-cd27227d6646 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.668468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08166b9e-98d7-479d-8778-cd27227d6646 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.669829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 765/1506] 10.222.0.69 () {52 vars in 830 bytes} [Thu Jul 28 06:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:26.673247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823d03b-a7cc-4a9f-81c4-e46247740f58 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.674525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3823d03b-a7cc-4a9f-81c4-e46247740f58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:26.773564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823d03b-a7cc-4a9f-81c4-e46247740f58 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.867246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.867883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.868165 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.868440 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] PATH_INFO: `/v3/users/829d523656084504b754a0a3bf2d5920` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.880600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] RBAC: Authorizing `identity:delete_user(user_id=829d523656084504b754a0a3bf2d5920)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:26.882188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:24:26.882188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Jul 28 06:24:26.882188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: Jul 28 06:24:26.883392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:26.893074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:27.054355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '829d523656084504b754a0a3bf2d5920'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:24:27.073132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '829d523656084504b754a0a3bf2d5920'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:24:27.086261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '829d523656084504b754a0a3bf2d5920'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:24:27.145425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '829d523656084504b754a0a3bf2d5920'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:24:27.145856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '829d523656084504b754a0a3bf2d5920'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:24:27.153510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 742/1507] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:24:26 2022] DELETE /identity/v3/users/829d523656084504b754a0a3bf2d5920 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:24:27.158284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22606be6-3f8c-4530-91f7-fa8bec8d6d6c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:27.158579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22606be6-3f8c-4530-91f7-fa8bec8d6d6c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:27.158869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22606be6-3f8c-4530-91f7-fa8bec8d6d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:27.215965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-22606be6-3f8c-4530-91f7-fa8bec8d6d6c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:27.217195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-22606be6-3f8c-4530-91f7-fa8bec8d6d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:27.293638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:27.308417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 766/1508] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:24:27.316939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:27.318903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:27.425549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b83be6-f08d-46d4-ab00-894c93ae37af admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:27.492889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:27.512864 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e3dbe-0a56-49ab-aa6d-65a88135485a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:27.513524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e3dbe-0a56-49ab-aa6d-65a88135485a service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:27.513806 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e3dbe-0a56-49ab-aa6d-65a88135485a service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:27.514082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e3dbe-0a56-49ab-aa6d-65a88135485a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:27.516072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c16e3dbe-0a56-49ab-aa6d-65a88135485a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:27.627633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e3dbe-0a56-49ab-aa6d-65a88135485a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:27.629008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e3dbe-0a56-49ab-aa6d-65a88135485a service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:27.647001 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 743/1509] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:24:28.105110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22606be6-3f8c-4530-91f7-fa8bec8d6d6c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:28.105110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-22606be6-3f8c-4530-91f7-fa8bec8d6d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:28.115566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22606be6-3f8c-4530-91f7-fa8bec8d6d6c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:28.217692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:28.218398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.218624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.218909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] PATH_INFO: `/v3/projects/9fbd04a98eb04e07a901d3bfd7047453` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.229097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fbd04a98eb04e07a901d3bfd7047453)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:28.230232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WARNING py.warnings [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:24:28.230232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: warnings.warn(msg) Jul 28 06:24:28.230232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: Jul 28 06:24:28.231663 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:28.376507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fbd04a98eb04e07a901d3bfd7047453', 'request_id': 'req-1141db14-efa1-45b8-bbdd-0069483b6d6a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:24:28.382806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fbd04a98eb04e07a901d3bfd7047453', 'request_id': 'req-1141db14-efa1-45b8-bbdd-0069483b6d6a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:24:28.436608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fbd04a98eb04e07a901d3bfd7047453', 'request_id': 'req-1141db14-efa1-45b8-bbdd-0069483b6d6a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:24:28.468389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] The token cache is being invalidate because project 9fbd04a98eb04e07a901d3bfd7047453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:24:28.468681 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:24:28.469263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1141db14-efa1-45b8-bbdd-0069483b6d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:24:28.470665 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 767/1510] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:24:28 2022] DELETE /identity/v3/projects/9fbd04a98eb04e07a901d3bfd7047453 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:24:37.644765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=62 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:38.466287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:06.176841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9064b-cfd7-4159-beb6-be39d9e1854c None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:06.177472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9064b-cfd7-4159-beb6-be39d9e1854c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:06.177580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9064b-cfd7-4159-beb6-be39d9e1854c None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:06.178882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:26:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:06.183809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b66db-ec91-4d97-bc3a-6ee96bdf89ac None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:06.184292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b66db-ec91-4d97-bc3a-6ee96bdf89ac None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:06.184391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b66db-ec91-4d97-bc3a-6ee96bdf89ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:06.265993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b28b66db-ec91-4d97-bc3a-6ee96bdf89ac None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:06.267221 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b28b66db-ec91-4d97-bc3a-6ee96bdf89ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:06.372370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:26:06.528852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9064b-cfd7-4159-beb6-be39d9e1854c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:06.533629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-52c9064b-cfd7-4159-beb6-be39d9e1854c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:06.655019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9064b-cfd7-4159-beb6-be39d9e1854c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:06.660996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b4cf5-7cb7-427b-b6f2-3b797ce18605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:06.661640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b4cf5-7cb7-427b-b6f2-3b797ce18605 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:06.661929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b4cf5-7cb7-427b-b6f2-3b797ce18605 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:06.662206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b4cf5-7cb7-427b-b6f2-3b797ce18605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:06.670229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b4cf5-7cb7-427b-b6f2-3b797ce18605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:06.671958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b4cf5-7cb7-427b-b6f2-3b797ce18605 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:06.681977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 745/1513] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:26:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:26:07.523930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db2f8c-2e97-4b51-b1fc-e280d531e70b None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:07.524511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db2f8c-2e97-4b51-b1fc-e280d531e70b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:07.524622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db2f8c-2e97-4b51-b1fc-e280d531e70b None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:07.526172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:26:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:07.531110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a044c-b7c7-4ccb-b049-e0944c93285c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:07.531573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a044c-b7c7-4ccb-b049-e0944c93285c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:07.531684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a044c-b7c7-4ccb-b049-e0944c93285c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:07.552430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-740a044c-b7c7-4ccb-b049-e0944c93285c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:07.553668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-740a044c-b7c7-4ccb-b049-e0944c93285c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:07.561566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:26:07.719521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db2f8c-2e97-4b51-b1fc-e280d531e70b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:07.735533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db2f8c-2e97-4b51-b1fc-e280d531e70b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:07.741208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3e4a7-1be8-45f7-ace2-f3ef0b7d6d55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:07.741864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3e4a7-1be8-45f7-ace2-f3ef0b7d6d55 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:07.742154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3e4a7-1be8-45f7-ace2-f3ef0b7d6d55 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:07.742435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3e4a7-1be8-45f7-ace2-f3ef0b7d6d55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:07.750951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3e4a7-1be8-45f7-ace2-f3ef0b7d6d55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:07.752548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3e4a7-1be8-45f7-ace2-f3ef0b7d6d55 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:07.762610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 770/1516] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:26:08.068626 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee280e07-74d3-4fbb-8560-c7d6753a6d21 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:08.068951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee280e07-74d3-4fbb-8560-c7d6753a6d21 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:08.069229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee280e07-74d3-4fbb-8560-c7d6753a6d21 None None] PATH_INFO: `/v3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:08.070662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 747/1517] 10.222.0.71 () {54 vars in 925 bytes} [Thu Jul 28 06:26:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:26:08.076900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904fc1d-9597-4f32-97fd-aa2faf598cd4 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:08.077194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904fc1d-9597-4f32-97fd-aa2faf598cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:08.077502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904fc1d-9597-4f32-97fd-aa2faf598cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:08.139563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-5904fc1d-9597-4f32-97fd-aa2faf598cd4 None None] MFA Rules not processed for user `244a8787df7641f2bac9b51213ddb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:08.140775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-5904fc1d-9597-4f32-97fd-aa2faf598cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:08.157999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:08.216747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 771/1518] 10.222.0.71 () {58 vars in 1015 bytes} [Thu Jul 28 06:26:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:26:12.231805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc027f61-6a5c-46f8-b948-0f648f933d0a None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:12.232356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc027f61-6a5c-46f8-b948-0f648f933d0a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:12.232356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc027f61-6a5c-46f8-b948-0f648f933d0a None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:12.233753 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 748/1519] 10.222.0.69 () {54 vars in 961 bytes} [Thu Jul 28 06:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:12.239289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065332d-b7ca-4f24-8054-63b90a5e6e41 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:12.239720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065332d-b7ca-4f24-8054-63b90a5e6e41 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:12.239885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065332d-b7ca-4f24-8054-63b90a5e6e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:12.259617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b065332d-b7ca-4f24-8054-63b90a5e6e41 None None] MFA Rules not processed for user `244a8787df7641f2bac9b51213ddb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:12.260830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b065332d-b7ca-4f24-8054-63b90a5e6e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:12.270306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 772/1520] 10.222.0.69 () {58 vars in 1058 bytes} [Thu Jul 28 06:26:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:26:12.277820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc027f61-6a5c-46f8-b948-0f648f933d0a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:12.291618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc027f61-6a5c-46f8-b948-0f648f933d0a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:12.296758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de66a3-ddd5-45a5-8398-4341828f699d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:12.297405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de66a3-ddd5-45a5-8398-4341828f699d service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:12.297693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de66a3-ddd5-45a5-8398-4341828f699d service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:12.298008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de66a3-ddd5-45a5-8398-4341828f699d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:12.307282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de66a3-ddd5-45a5-8398-4341828f699d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:12.308872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de66a3-ddd5-45a5-8398-4341828f699d service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:12.319658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 749/1521] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:26:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:26:18.994796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab152aa-477a-4a81-b54a-76b9483175c6 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:18.995349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab152aa-477a-4a81-b54a-76b9483175c6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:18.995349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab152aa-477a-4a81-b54a-76b9483175c6 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:18.996715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:19.001460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9865324-112c-41f9-aa1c-92a9a654b703 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:19.001943 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9865324-112c-41f9-aa1c-92a9a654b703 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:19.002105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9865324-112c-41f9-aa1c-92a9a654b703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:19.022402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b9865324-112c-41f9-aa1c-92a9a654b703 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:19.023621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b9865324-112c-41f9-aa1c-92a9a654b703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:19.031545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:26:19.194491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab152aa-477a-4a81-b54a-76b9483175c6 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:19.209172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab152aa-477a-4a81-b54a-76b9483175c6 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:19.214761 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fa3e2-5b90-42b9-975a-4a60f64e7cd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:19.215369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fa3e2-5b90-42b9-975a-4a60f64e7cd4 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:19.215664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fa3e2-5b90-42b9-975a-4a60f64e7cd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:19.215947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fa3e2-5b90-42b9-975a-4a60f64e7cd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:19.224753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fa3e2-5b90-42b9-975a-4a60f64e7cd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:19.226847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fa3e2-5b90-42b9-975a-4a60f64e7cd4 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:19.236527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 774/1524] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:26:20.074448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46029-47cf-47a4-b52d-bc4a8349a974 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:20.075007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46029-47cf-47a4-b52d-bc4a8349a974 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:20.075007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46029-47cf-47a4-b52d-bc4a8349a974 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:20.076389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:20.084565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0d92f-749b-43fa-925c-5b7d6c65ef34 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:20.085378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0d92f-749b-43fa-925c-5b7d6c65ef34 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:20.085378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0d92f-749b-43fa-925c-5b7d6c65ef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:20.108708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-5fa0d92f-749b-43fa-925c-5b7d6c65ef34 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:20.109976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-5fa0d92f-749b-43fa-925c-5b7d6c65ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:20.118078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:26:20.272775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b46029-47cf-47a4-b52d-bc4a8349a974 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:20.287908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b46029-47cf-47a4-b52d-bc4a8349a974 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:20.293470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c9794-7e18-42ef-bdbc-567e0220accf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:20.294078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c9794-7e18-42ef-bdbc-567e0220accf service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:20.294373 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c9794-7e18-42ef-bdbc-567e0220accf service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:20.294654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c9794-7e18-42ef-bdbc-567e0220accf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:20.303897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c9794-7e18-42ef-bdbc-567e0220accf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:20.305572 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c9794-7e18-42ef-bdbc-567e0220accf service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:20.315546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 752/1527] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:26:24.154018 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781667d1-f8bc-4723-a57d-12638d310030 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:24.154645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781667d1-f8bc-4723-a57d-12638d310030 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:24.154645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781667d1-f8bc-4723-a57d-12638d310030 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:24.155968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:24.161607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccdb7d-c552-4773-9d87-563e52d52d47 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:24.162109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccdb7d-c552-4773-9d87-563e52d52d47 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:24.162219 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccdb7d-c552-4773-9d87-563e52d52d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:24.183214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9cccdb7d-c552-4773-9d87-563e52d52d47 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:24.184356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9cccdb7d-c552-4773-9d87-563e52d52d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:24.192571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:26:24.272353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781667d1-f8bc-4723-a57d-12638d310030 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:24.274237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-781667d1-f8bc-4723-a57d-12638d310030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:24.356611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781667d1-f8bc-4723-a57d-12638d310030 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:24.360699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a475814-a740-4269-b9d4-b466a8c6c56f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:24.361360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a475814-a740-4269-b9d4-b466a8c6c56f service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:24.361653 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a475814-a740-4269-b9d4-b466a8c6c56f service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:24.361936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a475814-a740-4269-b9d4-b466a8c6c56f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:24.374261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a475814-a740-4269-b9d4-b466a8c6c56f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:24.375825 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a475814-a740-4269-b9d4-b466a8c6c56f service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:24.386673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 777/1530] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:26:26.453723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da9f3-8aa7-463e-9127-29399b74e049 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:26.454304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da9f3-8aa7-463e-9127-29399b74e049 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:26.454304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da9f3-8aa7-463e-9127-29399b74e049 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:26.455671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:26.461327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa747505-277f-4acd-aa88-c60dd61c394d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:26.461765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa747505-277f-4acd-aa88-c60dd61c394d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:26.461895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa747505-277f-4acd-aa88-c60dd61c394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:26.482263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-fa747505-277f-4acd-aa88-c60dd61c394d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:26.483457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-fa747505-277f-4acd-aa88-c60dd61c394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:26.491216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:26:26.527411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757da9f3-8aa7-463e-9127-29399b74e049 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:26.539294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757da9f3-8aa7-463e-9127-29399b74e049 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:26.543456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6c2b2-ce16-4a5b-a06f-41da21c221f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:26.544068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6c2b2-ce16-4a5b-a06f-41da21c221f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:26.544357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6c2b2-ce16-4a5b-a06f-41da21c221f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:26.544642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6c2b2-ce16-4a5b-a06f-41da21c221f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:26.553892 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6c2b2-ce16-4a5b-a06f-41da21c221f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:26.555329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6c2b2-ce16-4a5b-a06f-41da21c221f7 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:26.564840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 755/1533] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:26:30.144106 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c0bbd-9f3e-4d64-a99e-c88735a82d76 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:30.144686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c0bbd-9f3e-4d64-a99e-c88735a82d76 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:30.144686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c0bbd-9f3e-4d64-a99e-c88735a82d76 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:30.146077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:30.151450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600b3a-29a9-427b-b17e-5aa7f63bbcdc None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:30.151924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600b3a-29a9-427b-b17e-5aa7f63bbcdc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:30.152030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600b3a-29a9-427b-b17e-5aa7f63bbcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:30.159737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=81 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:30.171536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-50600b3a-29a9-427b-b17e-5aa7f63bbcdc None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:30.172749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-50600b3a-29a9-427b-b17e-5aa7f63bbcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:30.181073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:26:30.489675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c0bbd-9f3e-4d64-a99e-c88735a82d76 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:30.492060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-605c0bbd-9f3e-4d64-a99e-c88735a82d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:30.556358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:30.569264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c0bbd-9f3e-4d64-a99e-c88735a82d76 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:30.573150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f114b6-e392-418b-b659-0a80e8ed07f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:30.573839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f114b6-e392-418b-b659-0a80e8ed07f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:30.574121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f114b6-e392-418b-b659-0a80e8ed07f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:30.574425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f114b6-e392-418b-b659-0a80e8ed07f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:30.583185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f114b6-e392-418b-b659-0a80e8ed07f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:30.584603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f114b6-e392-418b-b659-0a80e8ed07f4 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:30.594228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 780/1536] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:26:32.651414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f26ce4-5733-4d51-b76c-c3c7f2175c46 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:32.651954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f26ce4-5733-4d51-b76c-c3c7f2175c46 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:32.651954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f26ce4-5733-4d51-b76c-c3c7f2175c46 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:32.653315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:32.658426 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e21a68-ae37-4339-b6a2-a76e4e80467e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:32.658854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e21a68-ae37-4339-b6a2-a76e4e80467e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:32.658990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e21a68-ae37-4339-b6a2-a76e4e80467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:32.680252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-23e21a68-ae37-4339-b6a2-a76e4e80467e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:32.681467 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-23e21a68-ae37-4339-b6a2-a76e4e80467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:32.689570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:26:32.995809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f26ce4-5733-4d51-b76c-c3c7f2175c46 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:32.997570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d5f26ce4-5733-4d51-b76c-c3c7f2175c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:33.011395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f26ce4-5733-4d51-b76c-c3c7f2175c46 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:33.014972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1fd46-45d4-4e53-844e-66d26243153c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:33.015587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1fd46-45d4-4e53-844e-66d26243153c service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:33.015876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1fd46-45d4-4e53-844e-66d26243153c service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:33.016231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1fd46-45d4-4e53-844e-66d26243153c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:33.025169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1fd46-45d4-4e53-844e-66d26243153c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:33.026709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1fd46-45d4-4e53-844e-66d26243153c service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:33.036057 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 758/1539] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:26:42.674949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=11 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:43.031948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=8 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:02.214739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfe7e8-428f-4226-8f5f-4c82312cb1e8 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.215919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfe7e8-428f-4226-8f5f-4c82312cb1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.216468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfe7e8-428f-4226-8f5f-4c82312cb1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.248450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-ffbfe7e8-428f-4226-8f5f-4c82312cb1e8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:02.250133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ffbfe7e8-428f-4226-8f5f-4c82312cb1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:02.261624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 782/1540] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:02.271951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e4616-aa26-478e-81bc-5902026ec23e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.273016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e4616-aa26-478e-81bc-5902026ec23e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.273569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e4616-aa26-478e-81bc-5902026ec23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.301799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7e5e4616-aa26-478e-81bc-5902026ec23e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:02.303391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7e5e4616-aa26-478e-81bc-5902026ec23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:02.317597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 759/1541] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:02.322819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfe7e8-428f-4226-8f5f-4c82312cb1e8 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:02.334478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfe7e8-428f-4226-8f5f-4c82312cb1e8 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:02.345866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16247e-5b86-4aad-b20a-30756e17eaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:02.346911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16247e-5b86-4aad-b20a-30756e17eaa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.347630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16247e-5b86-4aad-b20a-30756e17eaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.348386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16247e-5b86-4aad-b20a-30756e17eaa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.349360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16247e-5b86-4aad-b20a-30756e17eaa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:02.349897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16247e-5b86-4aad-b20a-30756e17eaa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:02.351746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16247e-5b86-4aad-b20a-30756e17eaa4 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:02.369341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789da7e-2acf-4e99-8c22-bd0c13b0257d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.370062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789da7e-2acf-4e99-8c22-bd0c13b0257d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.370643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789da7e-2acf-4e99-8c22-bd0c13b0257d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.373500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 783/1542] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:02.389999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16247e-5b86-4aad-b20a-30756e17eaa4 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:02.407165 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7789da7e-2acf-4e99-8c22-bd0c13b0257d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:02.408946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7789da7e-2acf-4e99-8c22-bd0c13b0257d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:02.418332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16247e-5b86-4aad-b20a-30756e17eaa4 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:02.423813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 760/1543] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:02.431431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3886a8-9a67-458c-a7e9-a1e0d8dc9fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:02.432353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3886a8-9a67-458c-a7e9-a1e0d8dc9fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.432824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3886a8-9a67-458c-a7e9-a1e0d8dc9fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.433045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b23ed-502f-4669-a0dd-a36c28c6cb92 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.433536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3886a8-9a67-458c-a7e9-a1e0d8dc9fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.433677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b23ed-502f-4669-a0dd-a36c28c6cb92 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.434032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b23ed-502f-4669-a0dd-a36c28c6cb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.434685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3886a8-9a67-458c-a7e9-a1e0d8dc9fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:02.435895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WARNING py.warnings [None req-ad3886a8-9a67-458c-a7e9-a1e0d8dc9fd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:28:02.435895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: warnings.warn(msg) Jul 28 06:28:02.435895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: Jul 28 06:28:02.437638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3886a8-9a67-458c-a7e9-a1e0d8dc9fd7 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:02.464541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0e8b23ed-502f-4669-a0dd-a36c28c6cb92 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:02.466281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0e8b23ed-502f-4669-a0dd-a36c28c6cb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:02.484545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 761/1544] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:02.495924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b23ed-502f-4669-a0dd-a36c28c6cb92 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:02.512911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b23ed-502f-4669-a0dd-a36c28c6cb92 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:02.519831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bff41-3dff-4ef8-b581-0f91d36b300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:02.520913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bff41-3dff-4ef8-b581-0f91d36b300b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.521465 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bff41-3dff-4ef8-b581-0f91d36b300b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.522010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bff41-3dff-4ef8-b581-0f91d36b300b admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.522937 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bff41-3dff-4ef8-b581-0f91d36b300b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:02.523458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bff41-3dff-4ef8-b581-0f91d36b300b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:02.524708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-202bff41-3dff-4ef8-b581-0f91d36b300b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:28:02.524708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Jul 28 06:28:02.524708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: Jul 28 06:28:02.529840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bff41-3dff-4ef8-b581-0f91d36b300b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:02.546249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 762/1545] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:28:02.547917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38206bf6-3add-4f50-9f5a-2c5bbd00b8bc None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.548485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38206bf6-3add-4f50-9f5a-2c5bbd00b8bc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.548997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38206bf6-3add-4f50-9f5a-2c5bbd00b8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.574607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-38206bf6-3add-4f50-9f5a-2c5bbd00b8bc None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:02.576198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-38206bf6-3add-4f50-9f5a-2c5bbd00b8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:02.580643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 784/1546] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/projects => generated 377 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:02.582410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3886a8-9a67-458c-a7e9-a1e0d8dc9fd7 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:02.589952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 763/1547] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:02.590970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38206bf6-3add-4f50-9f5a-2c5bbd00b8bc None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:02.608472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38206bf6-3add-4f50-9f5a-2c5bbd00b8bc None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:02.679831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77133f53-923c-400c-a383-d252eb6ce476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:02.680781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77133f53-923c-400c-a383-d252eb6ce476 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.681507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77133f53-923c-400c-a383-d252eb6ce476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.682059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77133f53-923c-400c-a383-d252eb6ce476 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.690298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77133f53-923c-400c-a383-d252eb6ce476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:02.690298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-77133f53-923c-400c-a383-d252eb6ce476 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:28:02.690298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Jul 28 06:28:02.690298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: Jul 28 06:28:02.692529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77133f53-923c-400c-a383-d252eb6ce476 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:02.736927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3886a8-9a67-458c-a7e9-a1e0d8dc9fd7 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:02.741001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc69d67-57ca-4cc0-a323-1aca67061119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:02.741684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc69d67-57ca-4cc0-a323-1aca67061119 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.741975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc69d67-57ca-4cc0-a323-1aca67061119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.742261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc69d67-57ca-4cc0-a323-1aca67061119 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.742952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc69d67-57ca-4cc0-a323-1aca67061119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:02.745175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc69d67-57ca-4cc0-a323-1aca67061119 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:02.795420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 764/1548] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:02.796702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdb1e2-4d3d-44c6-8e96-4649af3876b5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.796998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdb1e2-4d3d-44c6-8e96-4649af3876b5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.797285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdb1e2-4d3d-44c6-8e96-4649af3876b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.817227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e3cdb1e2-4d3d-44c6-8e96-4649af3876b5 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:02.818496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e3cdb1e2-4d3d-44c6-8e96-4649af3876b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:02.826597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 765/1549] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:02.827755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30683735-292d-4ca3-ab3b-ef5d8c6d4720 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.828048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30683735-292d-4ca3-ab3b-ef5d8c6d4720 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.828330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30683735-292d-4ca3-ab3b-ef5d8c6d4720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.848862 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-30683735-292d-4ca3-ab3b-ef5d8c6d4720 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:02.850126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-30683735-292d-4ca3-ab3b-ef5d8c6d4720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:02.858131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 766/1550] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:02.859312 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00dc6b-bf6f-4a98-a9f8-477ef8262f94 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.859604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00dc6b-bf6f-4a98-a9f8-477ef8262f94 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.859886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00dc6b-bf6f-4a98-a9f8-477ef8262f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.875764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 785/1551] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/projects => generated 377 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:02.882068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85e899-6812-4c05-b149-078dabcd4ee4 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.882068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85e899-6812-4c05-b149-078dabcd4ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.882068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85e899-6812-4c05-b149-078dabcd4ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.882407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4d00dc6b-bf6f-4a98-a9f8-477ef8262f94 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:02.882407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4d00dc6b-bf6f-4a98-a9f8-477ef8262f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:02.900923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-ea85e899-6812-4c05-b149-078dabcd4ee4 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:02.902222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ea85e899-6812-4c05-b149-078dabcd4ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:02.946000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 767/1552] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:02.947400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42213ed5-5bfc-4497-8c83-26a08d03b785 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.947836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42213ed5-5bfc-4497-8c83-26a08d03b785 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.948134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42213ed5-5bfc-4497-8c83-26a08d03b785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.965019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 786/1553] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:02.966216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85e899-6812-4c05-b149-078dabcd4ee4 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:02.974358 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-42213ed5-5bfc-4497-8c83-26a08d03b785 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:02.975582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-42213ed5-5bfc-4497-8c83-26a08d03b785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:02.977441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85e899-6812-4c05-b149-078dabcd4ee4 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:02.982102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50580e0a-1b3b-4680-943e-31dc92f762fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:02.982729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50580e0a-1b3b-4680-943e-31dc92f762fc admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.983033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50580e0a-1b3b-4680-943e-31dc92f762fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.983307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50580e0a-1b3b-4680-943e-31dc92f762fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.989598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 768/1554] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:02.989758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50580e0a-1b3b-4680-943e-31dc92f762fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:02.989859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e980-9df4-4456-8001-dece41b0147d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:02.989859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e980-9df4-4456-8001-dece41b0147d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:02.989859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e980-9df4-4456-8001-dece41b0147d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:02.990140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50580e0a-1b3b-4680-943e-31dc92f762fc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:02.999288 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 787/1555] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:03.000336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50580e0a-1b3b-4680-943e-31dc92f762fc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.004709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8f68e980-9df4-4456-8001-dece41b0147d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:03.005947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8f68e980-9df4-4456-8001-dece41b0147d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:03.010122 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50580e0a-1b3b-4680-943e-31dc92f762fc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.014048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 769/1556] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:03.015054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717ea8e-a56c-4faa-b2d8-3c56cf012215 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.015345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717ea8e-a56c-4faa-b2d8-3c56cf012215 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.015625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717ea8e-a56c-4faa-b2d8-3c56cf012215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.018293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34447bd-3e93-4a9e-8f24-4ad5465e908d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.019474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34447bd-3e93-4a9e-8f24-4ad5465e908d admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.019798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34447bd-3e93-4a9e-8f24-4ad5465e908d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.019995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34447bd-3e93-4a9e-8f24-4ad5465e908d admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.020700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34447bd-3e93-4a9e-8f24-4ad5465e908d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:03.021008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34447bd-3e93-4a9e-8f24-4ad5465e908d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.022832 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34447bd-3e93-4a9e-8f24-4ad5465e908d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.036163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9717ea8e-a56c-4faa-b2d8-3c56cf012215 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:03.037389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9717ea8e-a56c-4faa-b2d8-3c56cf012215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:03.037840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 788/1557] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:03.038645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34447bd-3e93-4a9e-8f24-4ad5465e908d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.045389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 770/1558] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:03.046837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7777-e99f-4230-b4e8-ffc42d3fcf1f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.046837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7777-e99f-4230-b4e8-ffc42d3fcf1f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.047043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7777-e99f-4230-b4e8-ffc42d3fcf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.050895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34447bd-3e93-4a9e-8f24-4ad5465e908d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.055749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a471af-a229-4a70-bc1a-d655c3bb1fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.056373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a471af-a229-4a70-bc1a-d655c3bb1fcb admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.056662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a471af-a229-4a70-bc1a-d655c3bb1fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.056948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a471af-a229-4a70-bc1a-d655c3bb1fcb admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.057664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a471af-a229-4a70-bc1a-d655c3bb1fcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.059590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a471af-a229-4a70-bc1a-d655c3bb1fcb admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.067591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-dd9e7777-e99f-4230-b4e8-ffc42d3fcf1f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:03.071656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dd9e7777-e99f-4230-b4e8-ffc42d3fcf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:03.071784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:03.078075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 771/1559] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:03.079290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a4a3-8a43-4292-9d76-c0bdbc6c77c7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.079576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a4a3-8a43-4292-9d76-c0bdbc6c77c7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.079853 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a4a3-8a43-4292-9d76-c0bdbc6c77c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.099721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-55f0a4a3-8a43-4292-9d76-c0bdbc6c77c7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:03.100917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-55f0a4a3-8a43-4292-9d76-c0bdbc6c77c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:03.108779 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 772/1560] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:03.109993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b5088-2fcc-4b52-8bd2-b862cd96383a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.110279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b5088-2fcc-4b52-8bd2-b862cd96383a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.110559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b5088-2fcc-4b52-8bd2-b862cd96383a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.118985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:03.130446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-638b5088-2fcc-4b52-8bd2-b862cd96383a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:03.131732 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-638b5088-2fcc-4b52-8bd2-b862cd96383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:03.140010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 773/1561] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:03.140933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b5088-2fcc-4b52-8bd2-b862cd96383a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.151076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b5088-2fcc-4b52-8bd2-b862cd96383a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.155820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962c785-fd6a-44e3-bd92-35def8976fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.156430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962c785-fd6a-44e3-bd92-35def8976fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.156726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962c785-fd6a-44e3-bd92-35def8976fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.157003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962c785-fd6a-44e3-bd92-35def8976fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.157748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962c785-fd6a-44e3-bd92-35def8976fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.159374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962c785-fd6a-44e3-bd92-35def8976fbf admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.169512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 774/1562] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:03.170848 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8571a08-b3b3-447e-ad09-afd5773ee557 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.171145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8571a08-b3b3-447e-ad09-afd5773ee557 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.171431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8571a08-b3b3-447e-ad09-afd5773ee557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.190638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f8571a08-b3b3-447e-ad09-afd5773ee557 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:03.192657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f8571a08-b3b3-447e-ad09-afd5773ee557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:03.200678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 775/1563] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:03.201601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8571a08-b3b3-447e-ad09-afd5773ee557 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.211939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8571a08-b3b3-447e-ad09-afd5773ee557 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.216519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d080ecd-38e9-492e-b407-02c47fb987fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.217194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d080ecd-38e9-492e-b407-02c47fb987fa admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.221061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d080ecd-38e9-492e-b407-02c47fb987fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.221061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d080ecd-38e9-492e-b407-02c47fb987fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.221061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d080ecd-38e9-492e-b407-02c47fb987fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.221061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d080ecd-38e9-492e-b407-02c47fb987fa admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.254609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 789/1564] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:03.255659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a471af-a229-4a70-bc1a-d655c3bb1fcb admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.265543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a471af-a229-4a70-bc1a-d655c3bb1fcb admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.273865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e809d-54cb-40f2-b6b8-268ffe55cca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.274494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e809d-54cb-40f2-b6b8-268ffe55cca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.274837 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e809d-54cb-40f2-b6b8-268ffe55cca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.275118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e809d-54cb-40f2-b6b8-268ffe55cca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.275828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e809d-54cb-40f2-b6b8-268ffe55cca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:03.276138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e809d-54cb-40f2-b6b8-268ffe55cca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.277779 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e809d-54cb-40f2-b6b8-268ffe55cca1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.291942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 790/1565] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:03.291942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e809d-54cb-40f2-b6b8-268ffe55cca1 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.302586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e809d-54cb-40f2-b6b8-268ffe55cca1 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.307278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0264a-6ee3-4942-95f2-71f2de56a2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.307738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 776/1566] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:03.308237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0264a-6ee3-4942-95f2-71f2de56a2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.308237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0264a-6ee3-4942-95f2-71f2de56a2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.308495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0264a-6ee3-4942-95f2-71f2de56a2f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.308718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d080ecd-38e9-492e-b407-02c47fb987fa admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.309260 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0264a-6ee3-4942-95f2-71f2de56a2f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:03.309638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0264a-6ee3-4942-95f2-71f2de56a2f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.311306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0264a-6ee3-4942-95f2-71f2de56a2f4 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.329489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 791/1567] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:03.329489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0264a-6ee3-4942-95f2-71f2de56a2f4 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.341143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0264a-6ee3-4942-95f2-71f2de56a2f4 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.376658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d080ecd-38e9-492e-b407-02c47fb987fa admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.382112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a7fe4-9a7d-43f7-b38f-9e5987531690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.382737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7fe4-9a7d-43f7-b38f-9e5987531690 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.383018 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7fe4-9a7d-43f7-b38f-9e5987531690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.383295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7fe4-9a7d-43f7-b38f-9e5987531690 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.383997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a7fe4-9a7d-43f7-b38f-9e5987531690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:03.384300 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a7fe4-9a7d-43f7-b38f-9e5987531690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.385968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a7fe4-9a7d-43f7-b38f-9e5987531690 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.398312 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 777/1568] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:03.399279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a7fe4-9a7d-43f7-b38f-9e5987531690 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.404238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a3574-5112-4b1c-83dc-21b5d961b78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.404909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a3574-5112-4b1c-83dc-21b5d961b78e admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.405201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a3574-5112-4b1c-83dc-21b5d961b78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.405531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a3574-5112-4b1c-83dc-21b5d961b78e admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.406277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a3574-5112-4b1c-83dc-21b5d961b78e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:03.406591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a3574-5112-4b1c-83dc-21b5d961b78e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.408216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a3574-5112-4b1c-83dc-21b5d961b78e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.410913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a7fe4-9a7d-43f7-b38f-9e5987531690 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.415946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19e82e-a05a-43b8-ae06-6dd9ff447122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.416608 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e82e-a05a-43b8-ae06-6dd9ff447122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.416883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e82e-a05a-43b8-ae06-6dd9ff447122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.417165 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e82e-a05a-43b8-ae06-6dd9ff447122 admin admin] PATH_INFO: `/v3/projects/58565c8c6b5743278fb155d39ed65e3d/users/eda0cfe646954a9a8476b346ac9fed72/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.420234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 792/1569] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:03.421163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a3574-5112-4b1c-83dc-21b5d961b78e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.432092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a3574-5112-4b1c-83dc-21b5d961b78e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.437279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e017c-be54-4b6b-9043-baf299b78991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.437934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e017c-be54-4b6b-9043-baf299b78991 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.438224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e017c-be54-4b6b-9043-baf299b78991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.438509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e017c-be54-4b6b-9043-baf299b78991 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.439215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e017c-be54-4b6b-9043-baf299b78991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:03.439530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e017c-be54-4b6b-9043-baf299b78991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.439793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19e82e-a05a-43b8-ae06-6dd9ff447122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58565c8c6b5743278fb155d39ed65e3d, user_id=eda0cfe646954a9a8476b346ac9fed72, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.441981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e017c-be54-4b6b-9043-baf299b78991 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.443161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19e82e-a05a-43b8-ae06-6dd9ff447122 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.454467 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 793/1570] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:03.458137 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e017c-be54-4b6b-9043-baf299b78991 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.465493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e017c-be54-4b6b-9043-baf299b78991 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.470524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ea212-dfa0-4945-8867-8feaa12e784d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.471119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea212-dfa0-4945-8867-8feaa12e784d admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.471410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea212-dfa0-4945-8867-8feaa12e784d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.471718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea212-dfa0-4945-8867-8feaa12e784d admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.473049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ea212-dfa0-4945-8867-8feaa12e784d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.474710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ea212-dfa0-4945-8867-8feaa12e784d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.484588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 794/1571] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:03.485384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ea212-dfa0-4945-8867-8feaa12e784d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.494796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ea212-dfa0-4945-8867-8feaa12e784d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.499484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972600b-e075-4a77-a249-8bbd2d24d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.502074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972600b-e075-4a77-a249-8bbd2d24d896 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.502074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972600b-e075-4a77-a249-8bbd2d24d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.502074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972600b-e075-4a77-a249-8bbd2d24d896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.502074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972600b-e075-4a77-a249-8bbd2d24d896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.503190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972600b-e075-4a77-a249-8bbd2d24d896 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.505198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 778/1572] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:03 2022] PUT /identity/v3/projects/58565c8c6b5743278fb155d39ed65e3d/users/eda0cfe646954a9a8476b346ac9fed72/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:03.505984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19e82e-a05a-43b8-ae06-6dd9ff447122 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.567640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19e82e-a05a-43b8-ae06-6dd9ff447122 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.572335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04e9ff-d7b3-44d9-9ecd-ef9a2bd16892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.572958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04e9ff-d7b3-44d9-9ecd-ef9a2bd16892 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.573263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04e9ff-d7b3-44d9-9ecd-ef9a2bd16892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.573564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04e9ff-d7b3-44d9-9ecd-ef9a2bd16892 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.574234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04e9ff-d7b3-44d9-9ecd-ef9a2bd16892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.576176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04e9ff-d7b3-44d9-9ecd-ef9a2bd16892 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.604211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 795/1573] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:03.605138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972600b-e075-4a77-a249-8bbd2d24d896 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.669846 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972600b-e075-4a77-a249-8bbd2d24d896 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.674734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2b6b8-b456-4b38-ab76-e765dd02cb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.675538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b6b8-b456-4b38-ab76-e765dd02cb2d admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.676046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b6b8-b456-4b38-ab76-e765dd02cb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.676343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b6b8-b456-4b38-ab76-e765dd02cb2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.677053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2b6b8-b456-4b38-ab76-e765dd02cb2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.679038 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2b6b8-b456-4b38-ab76-e765dd02cb2d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.681142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 779/1574] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:03.682122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04e9ff-d7b3-44d9-9ecd-ef9a2bd16892 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.692249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04e9ff-d7b3-44d9-9ecd-ef9a2bd16892 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.697588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1fb99-65af-44a8-acb4-93db5aee2767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.698224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1fb99-65af-44a8-acb4-93db5aee2767 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.698511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1fb99-65af-44a8-acb4-93db5aee2767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.698791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1fb99-65af-44a8-acb4-93db5aee2767 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.699498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1fb99-65af-44a8-acb4-93db5aee2767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.701481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1fb99-65af-44a8-acb4-93db5aee2767 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.718861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:03.768749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:03.775912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 796/1575] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:03.776986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2b6b8-b456-4b38-ab76-e765dd02cb2d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.797995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 780/1576] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:03.799492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1fb99-65af-44a8-acb4-93db5aee2767 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:03.840843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2b6b8-b456-4b38-ab76-e765dd02cb2d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.862217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1fb99-65af-44a8-acb4-93db5aee2767 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:03.866954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400fbb7-2623-4d4a-97bb-fd7df5617cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.867580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400fbb7-2623-4d4a-97bb-fd7df5617cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.867870 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400fbb7-2623-4d4a-97bb-fd7df5617cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.868151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400fbb7-2623-4d4a-97bb-fd7df5617cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.868828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400fbb7-2623-4d4a-97bb-fd7df5617cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.870778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400fbb7-2623-4d4a-97bb-fd7df5617cf1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.899172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7343b9-7996-49cb-a457-0f620125df98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:03.899804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7343b9-7996-49cb-a457-0f620125df98 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:03.900092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7343b9-7996-49cb-a457-0f620125df98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:03.900372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7343b9-7996-49cb-a457-0f620125df98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:03.901047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7343b9-7996-49cb-a457-0f620125df98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:03.903032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7343b9-7996-49cb-a457-0f620125df98 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:03.995899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 781/1577] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/projects => generated 393 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:03.996897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400fbb7-2623-4d4a-97bb-fd7df5617cf1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.010123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 797/1578] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:03 2022] POST /identity/v3/projects => generated 373 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:04.011294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2186a3-56bd-4cef-bc43-dea618430a59 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.011600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2186a3-56bd-4cef-bc43-dea618430a59 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.011894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2186a3-56bd-4cef-bc43-dea618430a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.032231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-fc2186a3-56bd-4cef-bc43-dea618430a59 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:04.033401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-fc2186a3-56bd-4cef-bc43-dea618430a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:04.073512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400fbb7-2623-4d4a-97bb-fd7df5617cf1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.094587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 798/1579] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:04.095778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2186a3-56bd-4cef-bc43-dea618430a59 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.106224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2186a3-56bd-4cef-bc43-dea618430a59 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.111077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a225-e74d-44f4-a640-d7c6bea5fbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.111701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a225-e74d-44f4-a640-d7c6bea5fbb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.111977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a225-e74d-44f4-a640-d7c6bea5fbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.112260 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a225-e74d-44f4-a640-d7c6bea5fbb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.113063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a225-e74d-44f4-a640-d7c6bea5fbb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.115206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a225-e74d-44f4-a640-d7c6bea5fbb2 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.148818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4754c-23c3-4ba5-95d4-65affcad66a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.149488 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4754c-23c3-4ba5-95d4-65affcad66a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.149777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4754c-23c3-4ba5-95d4-65affcad66a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.150052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4754c-23c3-4ba5-95d4-65affcad66a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.150745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4754c-23c3-4ba5-95d4-65affcad66a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.152347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4754c-23c3-4ba5-95d4-65affcad66a3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.162155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 782/1580] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.163131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4754c-23c3-4ba5-95d4-65affcad66a3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.172785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4754c-23c3-4ba5-95d4-65affcad66a3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.178243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76913227-de46-42bf-92c6-ca6f137692d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.178885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76913227-de46-42bf-92c6-ca6f137692d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.179175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76913227-de46-42bf-92c6-ca6f137692d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.179463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76913227-de46-42bf-92c6-ca6f137692d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.180155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76913227-de46-42bf-92c6-ca6f137692d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.181869 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76913227-de46-42bf-92c6-ca6f137692d9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.189823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 799/1581] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:04 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:04.192029 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a225-e74d-44f4-a640-d7c6bea5fbb2 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.194594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 783/1582] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.196979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76913227-de46-42bf-92c6-ca6f137692d9 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.203295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a225-e74d-44f4-a640-d7c6bea5fbb2 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.207130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76913227-de46-42bf-92c6-ca6f137692d9 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.208189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c1c1d-71c8-47bd-81de-e6bbceea1817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.208817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c1c1d-71c8-47bd-81de-e6bbceea1817 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.209256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c1c1d-71c8-47bd-81de-e6bbceea1817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.209673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c1c1d-71c8-47bd-81de-e6bbceea1817 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.210487 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c1c1d-71c8-47bd-81de-e6bbceea1817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.211933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebef11-7ba2-46d9-95a2-0dfeecc5dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.212497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c1c1d-71c8-47bd-81de-e6bbceea1817 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.212618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebef11-7ba2-46d9-95a2-0dfeecc5dc42 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.212830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebef11-7ba2-46d9-95a2-0dfeecc5dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.213117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebef11-7ba2-46d9-95a2-0dfeecc5dc42 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.213828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebef11-7ba2-46d9-95a2-0dfeecc5dc42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.215747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebef11-7ba2-46d9-95a2-0dfeecc5dc42 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.227075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:04.286878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:04.293249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 800/1583] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:04 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:04.294059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c1c1d-71c8-47bd-81de-e6bbceea1817 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.304926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c1c1d-71c8-47bd-81de-e6bbceea1817 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.309731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb5608-4c06-4edc-8907-e55dc6f7fef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.314861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb5608-4c06-4edc-8907-e55dc6f7fef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.314861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb5608-4c06-4edc-8907-e55dc6f7fef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.314861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb5608-4c06-4edc-8907-e55dc6f7fef0 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.314861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb5608-4c06-4edc-8907-e55dc6f7fef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.314861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb5608-4c06-4edc-8907-e55dc6f7fef0 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.348025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 784/1584] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:04 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:04.348786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebef11-7ba2-46d9-95a2-0dfeecc5dc42 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.358742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebef11-7ba2-46d9-95a2-0dfeecc5dc42 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.364311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380c02-bda3-4890-acc0-31bcb949eb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.364937 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380c02-bda3-4890-acc0-31bcb949eb46 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.365219 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380c02-bda3-4890-acc0-31bcb949eb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.365525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380c02-bda3-4890-acc0-31bcb949eb46 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.366945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf380c02-bda3-4890-acc0-31bcb949eb46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.368362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf380c02-bda3-4890-acc0-31bcb949eb46 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.411992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 801/1585] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:04 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:04.412844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb5608-4c06-4edc-8907-e55dc6f7fef0 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.468789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb5608-4c06-4edc-8907-e55dc6f7fef0 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.472973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2718ed-5800-47cd-a400-c4158a5a49d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.473631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2718ed-5800-47cd-a400-c4158a5a49d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.473925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2718ed-5800-47cd-a400-c4158a5a49d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.474224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2718ed-5800-47cd-a400-c4158a5a49d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.483759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2718ed-5800-47cd-a400-c4158a5a49d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.485245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2718ed-5800-47cd-a400-c4158a5a49d4 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.492347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 785/1586] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:04 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:04.498485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380c02-bda3-4890-acc0-31bcb949eb46 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.498613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 802/1587] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:04.498613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2718ed-5800-47cd-a400-c4158a5a49d4 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.506609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380c02-bda3-4890-acc0-31bcb949eb46 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.507992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2718ed-5800-47cd-a400-c4158a5a49d4 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.511542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4bd1e-71bf-4041-871d-807a2003eea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.512237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4bd1e-71bf-4041-871d-807a2003eea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.512524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4bd1e-71bf-4041-871d-807a2003eea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.512709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebf95a-d1db-4317-a9c2-40b7ca125e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.513150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4bd1e-71bf-4041-871d-807a2003eea2 admin admin] PATH_INFO: `/v3/projects/f6847d20741447f1b71a41e08fe90876/users/a326cc97673246639b474bc742679b8d/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.513534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebf95a-d1db-4317-a9c2-40b7ca125e9c admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.513820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebf95a-d1db-4317-a9c2-40b7ca125e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.514248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebf95a-d1db-4317-a9c2-40b7ca125e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.514987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebf95a-d1db-4317-a9c2-40b7ca125e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.516651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebf95a-d1db-4317-a9c2-40b7ca125e9c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.528118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 803/1588] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.528956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebf95a-d1db-4317-a9c2-40b7ca125e9c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.530054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4bd1e-71bf-4041-871d-807a2003eea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6847d20741447f1b71a41e08fe90876, user_id=a326cc97673246639b474bc742679b8d, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.533112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4bd1e-71bf-4041-871d-807a2003eea2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.541102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebf95a-d1db-4317-a9c2-40b7ca125e9c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.546868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6302e1-16a2-4ef5-b15c-f078a4183187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.547562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6302e1-16a2-4ef5-b15c-f078a4183187 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.547858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6302e1-16a2-4ef5-b15c-f078a4183187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.548163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6302e1-16a2-4ef5-b15c-f078a4183187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.548943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6302e1-16a2-4ef5-b15c-f078a4183187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.551292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6302e1-16a2-4ef5-b15c-f078a4183187 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.561642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 804/1589] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.562543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6302e1-16a2-4ef5-b15c-f078a4183187 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.572534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6302e1-16a2-4ef5-b15c-f078a4183187 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.581266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02f2ea-43a6-466f-84b7-328a75f81b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.581266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02f2ea-43a6-466f-84b7-328a75f81b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.581266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02f2ea-43a6-466f-84b7-328a75f81b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.581266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02f2ea-43a6-466f-84b7-328a75f81b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.581738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02f2ea-43a6-466f-84b7-328a75f81b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.581738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02f2ea-43a6-466f-84b7-328a75f81b8d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.592398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 805/1590] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.593406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02f2ea-43a6-466f-84b7-328a75f81b8d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.604777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02f2ea-43a6-466f-84b7-328a75f81b8d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.609489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4a32d-8222-45ba-be72-6729165c56da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.610103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4a32d-8222-45ba-be72-6729165c56da admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.610384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4a32d-8222-45ba-be72-6729165c56da admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.610662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4a32d-8222-45ba-be72-6729165c56da admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.611352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4a32d-8222-45ba-be72-6729165c56da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.613719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4a32d-8222-45ba-be72-6729165c56da admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.624121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 806/1591] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.625451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4a32d-8222-45ba-be72-6729165c56da admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.636474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4a32d-8222-45ba-be72-6729165c56da admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.657319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 786/1592] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:04 2022] PUT /identity/v3/projects/f6847d20741447f1b71a41e08fe90876/users/a326cc97673246639b474bc742679b8d/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:04.657319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4bd1e-71bf-4041-871d-807a2003eea2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.710351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95c78d-c1a1-4e12-9a96-1ae418d56fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.711175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95c78d-c1a1-4e12-9a96-1ae418d56fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.711758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95c78d-c1a1-4e12-9a96-1ae418d56fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.711888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95c78d-c1a1-4e12-9a96-1ae418d56fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.712498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95c78d-c1a1-4e12-9a96-1ae418d56fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.714141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95c78d-c1a1-4e12-9a96-1ae418d56fdc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.725297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 807/1593] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.727233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95c78d-c1a1-4e12-9a96-1ae418d56fdc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.729690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4bd1e-71bf-4041-871d-807a2003eea2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.734403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6baf5b-dbd8-4ca2-acd4-fe3f29d36731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.735028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6baf5b-dbd8-4ca2-acd4-fe3f29d36731 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.735308 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6baf5b-dbd8-4ca2-acd4-fe3f29d36731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.735583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6baf5b-dbd8-4ca2-acd4-fe3f29d36731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.736278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6baf5b-dbd8-4ca2-acd4-fe3f29d36731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.738212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6baf5b-dbd8-4ca2-acd4-fe3f29d36731 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.738413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95c78d-c1a1-4e12-9a96-1ae418d56fdc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.743624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ce4f1-96f6-43bc-96ee-0ede9719fbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.744723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ce4f1-96f6-43bc-96ee-0ede9719fbe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.744999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ce4f1-96f6-43bc-96ee-0ede9719fbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.745374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ce4f1-96f6-43bc-96ee-0ede9719fbe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.746257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ce4f1-96f6-43bc-96ee-0ede9719fbe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.747823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ce4f1-96f6-43bc-96ee-0ede9719fbe1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.748690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 787/1594] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.749524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6baf5b-dbd8-4ca2-acd4-fe3f29d36731 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.760324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6baf5b-dbd8-4ca2-acd4-fe3f29d36731 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.760536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 808/1595] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.761250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ce4f1-96f6-43bc-96ee-0ede9719fbe1 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.767708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe8651-917a-4bcb-a0c1-79db079f181d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.768388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe8651-917a-4bcb-a0c1-79db079f181d admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.768676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe8651-917a-4bcb-a0c1-79db079f181d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.769655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe8651-917a-4bcb-a0c1-79db079f181d admin admin] PATH_INFO: `/v3/projects/85bf0de611024c06aee30418f26d2f1c/users/61ad8186d91848bc8be42e3be51e9a75/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.771866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ce4f1-96f6-43bc-96ee-0ede9719fbe1 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.776603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69868a0-768c-4183-8b36-8b1ce794e6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.777340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69868a0-768c-4183-8b36-8b1ce794e6da admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.777548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69868a0-768c-4183-8b36-8b1ce794e6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.777806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69868a0-768c-4183-8b36-8b1ce794e6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.778510 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69868a0-768c-4183-8b36-8b1ce794e6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.780118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69868a0-768c-4183-8b36-8b1ce794e6da admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.787308 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe8651-917a-4bcb-a0c1-79db079f181d admin admin] RBAC: Authorizing `identity:create_grant(project_id=85bf0de611024c06aee30418f26d2f1c, user_id=61ad8186d91848bc8be42e3be51e9a75, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.790065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe8651-917a-4bcb-a0c1-79db079f181d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.794552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 809/1596] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.795390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69868a0-768c-4183-8b36-8b1ce794e6da admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.808410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69868a0-768c-4183-8b36-8b1ce794e6da admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.813239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719e8ed-c6f5-455c-bd79-5f525de5375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.815174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719e8ed-c6f5-455c-bd79-5f525de5375f admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.815174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719e8ed-c6f5-455c-bd79-5f525de5375f admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.815174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719e8ed-c6f5-455c-bd79-5f525de5375f admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.815174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719e8ed-c6f5-455c-bd79-5f525de5375f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.816909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719e8ed-c6f5-455c-bd79-5f525de5375f admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.828267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 810/1597] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.829664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbd11d-e1b5-4b6c-99b7-d86166d7dca4 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.829959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbd11d-e1b5-4b6c-99b7-d86166d7dca4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.830241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbd11d-e1b5-4b6c-99b7-d86166d7dca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.838081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:04.869968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-6bcbd11d-e1b5-4b6c-99b7-d86166d7dca4 None None] MFA Rules not processed for user `a326cc97673246639b474bc742679b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:04.875721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6bcbd11d-e1b5-4b6c-99b7-d86166d7dca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:04.884187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 788/1598] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:04 2022] PUT /identity/v3/projects/85bf0de611024c06aee30418f26d2f1c/users/61ad8186d91848bc8be42e3be51e9a75/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:04.885147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe8651-917a-4bcb-a0c1-79db079f181d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.964767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe8651-917a-4bcb-a0c1-79db079f181d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.976154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 811/1599] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:04.976335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fcb5e-92fe-4927-aaca-f0e32a89d6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.976461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbd11d-e1b5-4b6c-99b7-d86166d7dca4 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.976545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fcb5e-92fe-4927-aaca-f0e32a89d6ad admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.976545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fcb5e-92fe-4927-aaca-f0e32a89d6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.976545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fcb5e-92fe-4927-aaca-f0e32a89d6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.976762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fcb5e-92fe-4927-aaca-f0e32a89d6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:04.979532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fcb5e-92fe-4927-aaca-f0e32a89d6ad admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:04.987659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbd11d-e1b5-4b6c-99b7-d86166d7dca4 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:04.995250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 789/1600] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:04.995250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fcb5e-92fe-4927-aaca-f0e32a89d6ad admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:04.997745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e1bed-f7f9-4a4f-80d2-46c80fa02e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:04.998549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e1bed-f7f9-4a4f-80d2-46c80fa02e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:04.998840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e1bed-f7f9-4a4f-80d2-46c80fa02e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:04.999125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e1bed-f7f9-4a4f-80d2-46c80fa02e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:04.999836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e1bed-f7f9-4a4f-80d2-46c80fa02e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:05.001501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e1bed-f7f9-4a4f-80d2-46c80fa02e0b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:05.003634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fcb5e-92fe-4927-aaca-f0e32a89d6ad admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:05.009194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3085b-cd45-417b-a2d4-729b697b821d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:05.015380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3085b-cd45-417b-a2d4-729b697b821d admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.015380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3085b-cd45-417b-a2d4-729b697b821d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.015380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3085b-cd45-417b-a2d4-729b697b821d admin admin] PATH_INFO: `/v3/projects/a164b3cd6f444e17ae7ada0e30e62f54/users/6d4528589ffa478689e905d47e4c37ee/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.015671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 812/1601] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:05.015671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e1bed-f7f9-4a4f-80d2-46c80fa02e0b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:05.023381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e1bed-f7f9-4a4f-80d2-46c80fa02e0b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:05.027592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3085b-cd45-417b-a2d4-729b697b821d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a164b3cd6f444e17ae7ada0e30e62f54, user_id=6d4528589ffa478689e905d47e4c37ee, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:05.028346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c682ba-3fd8-4a3d-bb09-8e419adcd1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:05.029015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c682ba-3fd8-4a3d-bb09-8e419adcd1fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.029299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c682ba-3fd8-4a3d-bb09-8e419adcd1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.029604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c682ba-3fd8-4a3d-bb09-8e419adcd1fb admin admin] PATH_INFO: `/v3/projects/f7bae530a50e4c6ea97a654750ddf79f/users/0d268e37c8b64bcc9261a80bb0461edc/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.030517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3085b-cd45-417b-a2d4-729b697b821d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:05.047504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c682ba-3fd8-4a3d-bb09-8e419adcd1fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7bae530a50e4c6ea97a654750ddf79f, user_id=0d268e37c8b64bcc9261a80bb0461edc, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:05.048601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WARNING py.warnings [None req-f1c682ba-3fd8-4a3d-bb09-8e419adcd1fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:28:05.048601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: warnings.warn(msg) Jul 28 06:28:05.048601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: Jul 28 06:28:05.050959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c682ba-3fd8-4a3d-bb09-8e419adcd1fb admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:05.116939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:05.122903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 790/1602] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:04 2022] PUT /identity/v3/projects/a164b3cd6f444e17ae7ada0e30e62f54/users/6d4528589ffa478689e905d47e4c37ee/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:05.126776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3085b-cd45-417b-a2d4-729b697b821d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:05.189909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 813/1603] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:05 2022] PUT /identity/v3/projects/f7bae530a50e4c6ea97a654750ddf79f/users/0d268e37c8b64bcc9261a80bb0461edc/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:05.203476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89164b-0142-4641-8049-dd2636e5970e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.203476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89164b-0142-4641-8049-dd2636e5970e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.203476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89164b-0142-4641-8049-dd2636e5970e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.203970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3085b-cd45-417b-a2d4-729b697b821d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:05.212741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2a89164b-0142-4641-8049-dd2636e5970e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:05.214341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2a89164b-0142-4641-8049-dd2636e5970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:05.256994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb7ead-5a9d-4d10-9bf6-dd2ae5a5f9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:05.257704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb7ead-5a9d-4d10-9bf6-dd2ae5a5f9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.257990 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb7ead-5a9d-4d10-9bf6-dd2ae5a5f9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.258272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb7ead-5a9d-4d10-9bf6-dd2ae5a5f9ef admin admin] PATH_INFO: `/v3/projects/fc0bbee6f2334c599f7756b95c9d46dc/users/5cc5c21a50d5452a8b9b9829f69ad0d2/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.273801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb7ead-5a9d-4d10-9bf6-dd2ae5a5f9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0bbee6f2334c599f7756b95c9d46dc, user_id=5cc5c21a50d5452a8b9b9829f69ad0d2, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:05.276235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 814/1604] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:05.279718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89164b-0142-4641-8049-dd2636e5970e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:05.279820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb7ead-5a9d-4d10-9bf6-dd2ae5a5f9ef admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:05.289976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89164b-0142-4641-8049-dd2636e5970e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:05.295033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0a0d-1d06-41eb-a726-68fdd46018bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:05.295753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0a0d-1d06-41eb-a726-68fdd46018bc admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.296038 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0a0d-1d06-41eb-a726-68fdd46018bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.296322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0a0d-1d06-41eb-a726-68fdd46018bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.297012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e0a0d-1d06-41eb-a726-68fdd46018bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:05.299131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e0a0d-1d06-41eb-a726-68fdd46018bc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:05.373347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 791/1605] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:05 2022] PUT /identity/v3/projects/fc0bbee6f2334c599f7756b95c9d46dc/users/5cc5c21a50d5452a8b9b9829f69ad0d2/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:05.375189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb7ead-5a9d-4d10-9bf6-dd2ae5a5f9ef admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:05.400357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 815/1606] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:05 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:05.401659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0a0d-1d06-41eb-a726-68fdd46018bc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:05.447229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:05.458452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb7ead-5a9d-4d10-9bf6-dd2ae5a5f9ef admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:05.472362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0a0d-1d06-41eb-a726-68fdd46018bc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:05.477412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607de4d-dcb2-4c72-93c9-724b49490638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:05.478100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607de4d-dcb2-4c72-93c9-724b49490638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.478389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607de4d-dcb2-4c72-93c9-724b49490638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.478675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607de4d-dcb2-4c72-93c9-724b49490638 admin admin] PATH_INFO: `/v3/projects/7effa0ed8b924e2c9e305aec5c5bd79a/users/2c64f0088817460394490a1bed2c258e/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.492670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:05.495345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607de4d-dcb2-4c72-93c9-724b49490638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7effa0ed8b924e2c9e305aec5c5bd79a, user_id=2c64f0088817460394490a1bed2c258e, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:05.502702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607de4d-dcb2-4c72-93c9-724b49490638 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:05.521147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20324f95-f693-4e61-a925-741d4d0ad251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:05.521883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20324f95-f693-4e61-a925-741d4d0ad251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.522165 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20324f95-f693-4e61-a925-741d4d0ad251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.522445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20324f95-f693-4e61-a925-741d4d0ad251 admin admin] PATH_INFO: `/v3/projects/cd88554db98441c99e0b09b446c24b47/users/86a5553819d949ffbc64b7a388942baf/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.538660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20324f95-f693-4e61-a925-741d4d0ad251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd88554db98441c99e0b09b446c24b47, user_id=86a5553819d949ffbc64b7a388942baf, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:05.541709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20324f95-f693-4e61-a925-741d4d0ad251 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:05.578560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 816/1607] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:05 2022] PUT /identity/v3/projects/7effa0ed8b924e2c9e305aec5c5bd79a/users/2c64f0088817460394490a1bed2c258e/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:05.578560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46e5fc-61cf-45b7-93d5-36b0f2057fa8 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.578560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46e5fc-61cf-45b7-93d5-36b0f2057fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.578560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46e5fc-61cf-45b7-93d5-36b0f2057fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.600699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-3d46e5fc-61cf-45b7-93d5-36b0f2057fa8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:05.602316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3d46e5fc-61cf-45b7-93d5-36b0f2057fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:05.638919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 792/1608] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:05 2022] PUT /identity/v3/projects/cd88554db98441c99e0b09b446c24b47/users/86a5553819d949ffbc64b7a388942baf/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:05.640677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d74ff-ea94-4c96-b91a-77d5ca7f687d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.640983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d74ff-ea94-4c96-b91a-77d5ca7f687d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.641267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d74ff-ea94-4c96-b91a-77d5ca7f687d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.674805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c64d74ff-ea94-4c96-b91a-77d5ca7f687d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:05.676555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c64d74ff-ea94-4c96-b91a-77d5ca7f687d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:05.680190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 817/1609] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:05.681160 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46e5fc-61cf-45b7-93d5-36b0f2057fa8 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:05.682652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3d46e5fc-61cf-45b7-93d5-36b0f2057fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:05.744545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46e5fc-61cf-45b7-93d5-36b0f2057fa8 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:05.744797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 793/1610] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:05.745883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba37bb-8a69-4a7d-b0af-b1819e724c24 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.746173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba37bb-8a69-4a7d-b0af-b1819e724c24 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.746522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba37bb-8a69-4a7d-b0af-b1819e724c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.775056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-47ba37bb-8a69-4a7d-b0af-b1819e724c24 None None] MFA Rules not processed for user `5cc5c21a50d5452a8b9b9829f69ad0d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:05.776269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-47ba37bb-8a69-4a7d-b0af-b1819e724c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:05.796842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab054aed-a65d-4732-98da-af88ef41ca0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:05.803294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054aed-a65d-4732-98da-af88ef41ca0e service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.803294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054aed-a65d-4732-98da-af88ef41ca0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.803294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054aed-a65d-4732-98da-af88ef41ca0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.807649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab054aed-a65d-4732-98da-af88ef41ca0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:05.809099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab054aed-a65d-4732-98da-af88ef41ca0e service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:05.819297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 818/1611] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:05.820332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab054aed-a65d-4732-98da-af88ef41ca0e service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:05.829465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab054aed-a65d-4732-98da-af88ef41ca0e service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:05.836573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d296ae4-9599-48b8-98ee-909a01668197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:05.839831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d296ae4-9599-48b8-98ee-909a01668197 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.839831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d296ae4-9599-48b8-98ee-909a01668197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.839831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d296ae4-9599-48b8-98ee-909a01668197 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.839831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d296ae4-9599-48b8-98ee-909a01668197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:05.840602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d296ae4-9599-48b8-98ee-909a01668197 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:05.846816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 794/1612] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:05.848718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3a8c4-dfe4-4103-ad3b-9336309cddd9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.849075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3a8c4-dfe4-4103-ad3b-9336309cddd9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.849442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3a8c4-dfe4-4103-ad3b-9336309cddd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.889563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-eed3a8c4-dfe4-4103-ad3b-9336309cddd9 None None] MFA Rules not processed for user `2c64f0088817460394490a1bed2c258e`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:05.891631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-eed3a8c4-dfe4-4103-ad3b-9336309cddd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:05.930608 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:05.951297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:05.960547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 819/1613] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:05 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:05.963302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71b98e-97d0-48e0-b4fd-2d4deed0fe91 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.963753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71b98e-97d0-48e0-b4fd-2d4deed0fe91 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.964190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71b98e-97d0-48e0-b4fd-2d4deed0fe91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.965450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 795/1614] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:05.966381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3a8c4-dfe4-4103-ad3b-9336309cddd9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:05.981770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3a8c4-dfe4-4103-ad3b-9336309cddd9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:05.986920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e3e32-94d7-432f-9cc0-70382fb8c9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:05.988169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3e32-94d7-432f-9cc0-70382fb8c9bf service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:05.988596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3e32-94d7-432f-9cc0-70382fb8c9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:05.988932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3e32-94d7-432f-9cc0-70382fb8c9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:05.989690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-fb71b98e-97d0-48e0-b4fd-2d4deed0fe91 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:05.992527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-fb71b98e-97d0-48e0-b4fd-2d4deed0fe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:06.002274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e3e32-94d7-432f-9cc0-70382fb8c9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.006421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 820/1615] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:06.006421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71b98e-97d0-48e0-b4fd-2d4deed0fe91 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.006680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e3e32-94d7-432f-9cc0-70382fb8c9bf service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.018268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71b98e-97d0-48e0-b4fd-2d4deed0fe91 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.019858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 796/1616] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:06.021614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d30369-9086-40fe-9b3d-db627b5dfc85 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.022172 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d30369-9086-40fe-9b3d-db627b5dfc85 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.022657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d30369-9086-40fe-9b3d-db627b5dfc85 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.024506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 797/1617] 10.222.0.69 () {54 vars in 962 bytes} [Thu Jul 28 06:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:06.025169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc7552-e753-4f55-add2-d2fa2d67c699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.025537 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d30369-9086-40fe-9b3d-db627b5dfc85 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.026369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc7552-e753-4f55-add2-d2fa2d67c699 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.026869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc7552-e753-4f55-add2-d2fa2d67c699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.027862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc7552-e753-4f55-add2-d2fa2d67c699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.039977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc7552-e753-4f55-add2-d2fa2d67c699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.041860 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d30369-9086-40fe-9b3d-db627b5dfc85 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.042142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc7552-e753-4f55-add2-d2fa2d67c699 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.047027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8b732-e347-400b-b1b5-4b7accd96898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.047875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8b732-e347-400b-b1b5-4b7accd96898 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.048266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8b732-e347-400b-b1b5-4b7accd96898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.048686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8b732-e347-400b-b1b5-4b7accd96898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.049948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8b732-e347-400b-b1b5-4b7accd96898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.052173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8b732-e347-400b-b1b5-4b7accd96898 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.058613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 821/1618] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:06.059585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc7552-e753-4f55-add2-d2fa2d67c699 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.068323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 798/1619] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:06.070358 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90630d6f-b88b-40f3-91e0-fc14df2effd2 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.070669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90630d6f-b88b-40f3-91e0-fc14df2effd2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.071226 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90630d6f-b88b-40f3-91e0-fc14df2effd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.078807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc7552-e753-4f55-add2-d2fa2d67c699 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.078807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1db97-20cc-403a-b49c-3a9bc6defac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.079387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1db97-20cc-403a-b49c-3a9bc6defac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.079769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1db97-20cc-403a-b49c-3a9bc6defac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.079975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1db97-20cc-403a-b49c-3a9bc6defac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.080701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1db97-20cc-403a-b49c-3a9bc6defac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.082963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1db97-20cc-403a-b49c-3a9bc6defac6 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.135114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-90630d6f-b88b-40f3-91e0-fc14df2effd2 None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:06.137627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-90630d6f-b88b-40f3-91e0-fc14df2effd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:06.244430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 799/1620] 10.222.0.69 () {58 vars in 1059 bytes} [Thu Jul 28 06:28:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:06.245339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90630d6f-b88b-40f3-91e0-fc14df2effd2 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.260379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90630d6f-b88b-40f3-91e0-fc14df2effd2 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.264838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b4910-9f8b-46cb-a1d5-fc4cd3eda5aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.265672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b4910-9f8b-46cb-a1d5-fc4cd3eda5aa service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.266023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b4910-9f8b-46cb-a1d5-fc4cd3eda5aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.266650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b4910-9f8b-46cb-a1d5-fc4cd3eda5aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.280052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 822/1621] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:06 2022] POST /identity/v3/projects => generated 369 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:06.281045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1db97-20cc-403a-b49c-3a9bc6defac6 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.281339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b4910-9f8b-46cb-a1d5-fc4cd3eda5aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.291050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b4910-9f8b-46cb-a1d5-fc4cd3eda5aa service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.300534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 800/1622] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:06.302542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b4910-9f8b-46cb-a1d5-fc4cd3eda5aa service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.319409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b4910-9f8b-46cb-a1d5-fc4cd3eda5aa service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.407042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1db97-20cc-403a-b49c-3a9bc6defac6 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.414698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df06fbc-b117-4e74-8faa-5a4e33569a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.415445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df06fbc-b117-4e74-8faa-5a4e33569a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.415958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df06fbc-b117-4e74-8faa-5a4e33569a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.415958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df06fbc-b117-4e74-8faa-5a4e33569a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.416784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df06fbc-b117-4e74-8faa-5a4e33569a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.418573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df06fbc-b117-4e74-8faa-5a4e33569a64 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.425991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24d36d-3efe-4b21-bc29-03aa9aed6fb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.425991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24d36d-3efe-4b21-bc29-03aa9aed6fb9 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.425991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24d36d-3efe-4b21-bc29-03aa9aed6fb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.425991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24d36d-3efe-4b21-bc29-03aa9aed6fb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.436888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 823/1623] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:06.436888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df06fbc-b117-4e74-8faa-5a4e33569a64 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.448463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df06fbc-b117-4e74-8faa-5a4e33569a64 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.454240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5535266-9a75-4f88-84a9-3edd7237dfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.455011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5535266-9a75-4f88-84a9-3edd7237dfc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.455299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5535266-9a75-4f88-84a9-3edd7237dfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.455576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5535266-9a75-4f88-84a9-3edd7237dfc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.456887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5535266-9a75-4f88-84a9-3edd7237dfc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.460073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5535266-9a75-4f88-84a9-3edd7237dfc2 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.478719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:06.508665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24d36d-3efe-4b21-bc29-03aa9aed6fb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.508665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24d36d-3efe-4b21-bc29-03aa9aed6fb9 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.528019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 801/1624] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:06.530079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24d36d-3efe-4b21-bc29-03aa9aed6fb9 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.558333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24d36d-3efe-4b21-bc29-03aa9aed6fb9 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.570115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5db759-56ea-480f-aa52-ddaaac73f8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.571008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5db759-56ea-480f-aa52-ddaaac73f8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.571510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5db759-56ea-480f-aa52-ddaaac73f8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.571954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5db759-56ea-480f-aa52-ddaaac73f8bd admin admin] PATH_INFO: `/v3/projects/6bcab5b6799b45079b12f9f4fd4f6e4e/users/5a9e2a9962de4a7897ac0e3abde497d2/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.598141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 824/1625] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:06 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:06.603449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5535266-9a75-4f88-84a9-3edd7237dfc2 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.609030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5db759-56ea-480f-aa52-ddaaac73f8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bcab5b6799b45079b12f9f4fd4f6e4e, user_id=5a9e2a9962de4a7897ac0e3abde497d2, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.612061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5db759-56ea-480f-aa52-ddaaac73f8bd admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.631470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5535266-9a75-4f88-84a9-3edd7237dfc2 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.653243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de4947-12a8-42e1-bc75-5398c57390fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.653243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de4947-12a8-42e1-bc75-5398c57390fc admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.653243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de4947-12a8-42e1-bc75-5398c57390fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.653243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de4947-12a8-42e1-bc75-5398c57390fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.653896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de4947-12a8-42e1-bc75-5398c57390fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.655547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de4947-12a8-42e1-bc75-5398c57390fc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.681825 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 825/1626] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:06.686639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de4947-12a8-42e1-bc75-5398c57390fc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.706775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 802/1627] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:06 2022] PUT /identity/v3/projects/6bcab5b6799b45079b12f9f4fd4f6e4e/users/5a9e2a9962de4a7897ac0e3abde497d2/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:06.710916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de4947-12a8-42e1-bc75-5398c57390fc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.720289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387ff0c-9e87-49c3-b13b-915fc40ef097 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.720844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387ff0c-9e87-49c3-b13b-915fc40ef097 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.721428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387ff0c-9e87-49c3-b13b-915fc40ef097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.760892 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0387ff0c-9e87-49c3-b13b-915fc40ef097 None None] MFA Rules not processed for user `5a9e2a9962de4a7897ac0e3abde497d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:06.765927 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0387ff0c-9e87-49c3-b13b-915fc40ef097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:06.864920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e489d-b360-43f2-9d39-520abacfa877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.865595 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e489d-b360-43f2-9d39-520abacfa877 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.865887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e489d-b360-43f2-9d39-520abacfa877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.866171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e489d-b360-43f2-9d39-520abacfa877 admin admin] PATH_INFO: `/v3/projects/14adc84a39404a9d8776354905729b28/users/0ef65b9e1d9c457784c6f70054f74509/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.905050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 803/1628] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:06.907620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e489d-b360-43f2-9d39-520abacfa877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14adc84a39404a9d8776354905729b28, user_id=0ef65b9e1d9c457784c6f70054f74509, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.911896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387ff0c-9e87-49c3-b13b-915fc40ef097 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:06.913120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e489d-b360-43f2-9d39-520abacfa877 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:06.931824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387ff0c-9e87-49c3-b13b-915fc40ef097 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:06.937271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4053a2e-0f54-4883-ba3b-d9c412fe3d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:06.938124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4053a2e-0f54-4883-ba3b-d9c412fe3d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:06.938508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4053a2e-0f54-4883-ba3b-d9c412fe3d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:06.938898 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4053a2e-0f54-4883-ba3b-d9c412fe3d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:06.944245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4053a2e-0f54-4883-ba3b-d9c412fe3d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:06.946793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4053a2e-0f54-4883-ba3b-d9c412fe3d87 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:07.046761 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 826/1629] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:06 2022] PUT /identity/v3/projects/14adc84a39404a9d8776354905729b28/users/0ef65b9e1d9c457784c6f70054f74509/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:07.047857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e489d-b360-43f2-9d39-520abacfa877 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:07.076538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:07.101672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 804/1630] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:06 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:07.106402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4053a2e-0f54-4883-ba3b-d9c412fe3d87 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:07.204554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e489d-b360-43f2-9d39-520abacfa877 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:07.230765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4053a2e-0f54-4883-ba3b-d9c412fe3d87 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:07.244916 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758652e8-7463-41eb-9176-a073909d0cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:07.245639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758652e8-7463-41eb-9176-a073909d0cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:07.246451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758652e8-7463-41eb-9176-a073909d0cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:07.246741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758652e8-7463-41eb-9176-a073909d0cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:07.247612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758652e8-7463-41eb-9176-a073909d0cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:07.251683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758652e8-7463-41eb-9176-a073909d0cb2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:07.269043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:07.389065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa7d24-5e3e-40a4-a550-ddb832ffbc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:07.389065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa7d24-5e3e-40a4-a550-ddb832ffbc82 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:07.389065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa7d24-5e3e-40a4-a550-ddb832ffbc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:07.389065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa7d24-5e3e-40a4-a550-ddb832ffbc82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:07.401738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa7d24-5e3e-40a4-a550-ddb832ffbc82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:07.401738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa7d24-5e3e-40a4-a550-ddb832ffbc82 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:07.404804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 805/1631] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:07 2022] POST /identity/v3/users => generated 296 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:07.408992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758652e8-7463-41eb-9176-a073909d0cb2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:07.421780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 827/1632] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:07.426635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc990b-3536-4b6b-926c-c264cb3928c8 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:07.427083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc990b-3536-4b6b-926c-c264cb3928c8 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:07.427831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc990b-3536-4b6b-926c-c264cb3928c8 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:07.433962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 828/1633] 10.222.0.69 () {54 vars in 962 bytes} [Thu Jul 28 06:28:07 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:07.435237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc990b-3536-4b6b-926c-c264cb3928c8 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:07.440030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758652e8-7463-41eb-9176-a073909d0cb2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:07.452843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de96c6-0116-4295-aa6e-9a0341242471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:07.454614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de96c6-0116-4295-aa6e-9a0341242471 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:07.454911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de96c6-0116-4295-aa6e-9a0341242471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:07.455326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de96c6-0116-4295-aa6e-9a0341242471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:07.455969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de96c6-0116-4295-aa6e-9a0341242471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:07.460792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de96c6-0116-4295-aa6e-9a0341242471 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:07.469619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc990b-3536-4b6b-926c-c264cb3928c8 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:07.483569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940d4c8-0422-44d6-bac0-f58f987dfbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:07.486551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940d4c8-0422-44d6-bac0-f58f987dfbc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:07.486842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940d4c8-0422-44d6-bac0-f58f987dfbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:07.487079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940d4c8-0422-44d6-bac0-f58f987dfbc2 admin admin] PATH_INFO: `/v3/projects/14adc84a39404a9d8776354905729b28/users/0ef65b9e1d9c457784c6f70054f74509/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:07.492987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 806/1634] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:28:07.495526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a940d4c8-0422-44d6-bac0-f58f987dfbc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14adc84a39404a9d8776354905729b28, user_id=0ef65b9e1d9c457784c6f70054f74509, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:07.497371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082afe2-4f9b-49ef-88d7-84610dfc89aa None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:07.497371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082afe2-4f9b-49ef-88d7-84610dfc89aa None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:07.508012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082afe2-4f9b-49ef-88d7-84610dfc89aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:07.513680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a940d4c8-0422-44d6-bac0-f58f987dfbc2 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:07.547930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 829/1635] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:07 2022] PUT /identity/v3/projects/14adc84a39404a9d8776354905729b28/users/0ef65b9e1d9c457784c6f70054f74509/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:07.563384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940d4c8-0422-44d6-bac0-f58f987dfbc2 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:07.576835 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c082afe2-4f9b-49ef-88d7-84610dfc89aa None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:07.579973 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c082afe2-4f9b-49ef-88d7-84610dfc89aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:07.666403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940d4c8-0422-44d6-bac0-f58f987dfbc2 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:07.674574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb36fad-2927-4016-85e7-5a07dd2d6494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:07.677325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb36fad-2927-4016-85e7-5a07dd2d6494 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:07.677325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb36fad-2927-4016-85e7-5a07dd2d6494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:07.677325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb36fad-2927-4016-85e7-5a07dd2d6494 admin admin] PATH_INFO: `/v3/projects/91ba5e94d5f141b793bc5f59793b9274/users/c76aa12ac4744957ad7292b005cb3df7/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:07.681648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 807/1636] 10.222.0.69 () {58 vars in 1059 bytes} [Thu Jul 28 06:28:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:07.683523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d862f0-e9b0-48e7-a06c-7613c8023817 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:07.684009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d862f0-e9b0-48e7-a06c-7613c8023817 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:07.684455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d862f0-e9b0-48e7-a06c-7613c8023817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:07.699010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb36fad-2927-4016-85e7-5a07dd2d6494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ba5e94d5f141b793bc5f59793b9274, user_id=c76aa12ac4744957ad7292b005cb3df7, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:07.702661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb36fad-2927-4016-85e7-5a07dd2d6494 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:07.726066 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e8d862f0-e9b0-48e7-a06c-7613c8023817 None None] MFA Rules not processed for user `0ef65b9e1d9c457784c6f70054f74509`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:07.727527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e8d862f0-e9b0-48e7-a06c-7613c8023817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:07.821674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 808/1637] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:07 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:07.822841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88131cf3-1fcc-48c6-bdc7-9beb9ba4304b None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:07.823145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88131cf3-1fcc-48c6-bdc7-9beb9ba4304b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:07.823436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88131cf3-1fcc-48c6-bdc7-9beb9ba4304b None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:07.826749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 809/1638] 10.222.0.69 () {54 vars in 912 bytes} [Thu Jul 28 06:28:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:07.841151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deceaa-b46f-4410-a078-788c060b41d7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:07.842720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deceaa-b46f-4410-a078-788c060b41d7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:07.842720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deceaa-b46f-4410-a078-788c060b41d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:07.848750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 830/1639] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:07 2022] PUT /identity/v3/projects/91ba5e94d5f141b793bc5f59793b9274/users/c76aa12ac4744957ad7292b005cb3df7/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:07.852904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb36fad-2927-4016-85e7-5a07dd2d6494 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:07.870934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-42deceaa-b46f-4410-a078-788c060b41d7 None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:07.872226 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-42deceaa-b46f-4410-a078-788c060b41d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:07.957604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 810/1640] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:28:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:07.957604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deceaa-b46f-4410-a078-788c060b41d7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:07.999891 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb36fad-2927-4016-85e7-5a07dd2d6494 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:08.009286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03293530-8eb9-4f2c-b415-b840c44f19e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:08.009980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293530-8eb9-4f2c-b415-b840c44f19e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:08.010264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293530-8eb9-4f2c-b415-b840c44f19e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:08.010542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293530-8eb9-4f2c-b415-b840c44f19e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:08.073456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deceaa-b46f-4410-a078-788c060b41d7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:08.080208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e978-8e08-44e9-bab5-69cbe6832045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:08.081362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e978-8e08-44e9-bab5-69cbe6832045 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:08.089443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e978-8e08-44e9-bab5-69cbe6832045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:08.089443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e978-8e08-44e9-bab5-69cbe6832045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:08.089443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e978-8e08-44e9-bab5-69cbe6832045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:08.089443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e978-8e08-44e9-bab5-69cbe6832045 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:08.102584 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 811/1641] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:08.102584 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e978-8e08-44e9-bab5-69cbe6832045 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:08.115315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e978-8e08-44e9-bab5-69cbe6832045 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:08.125706 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c971c-d5f8-4753-97ef-82e852ec81a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:08.126571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c971c-d5f8-4753-97ef-82e852ec81a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:08.126963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c971c-d5f8-4753-97ef-82e852ec81a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:08.128734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c971c-d5f8-4753-97ef-82e852ec81a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:08.131553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03293530-8eb9-4f2c-b415-b840c44f19e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:08.133190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03293530-8eb9-4f2c-b415-b840c44f19e7 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:08.142840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c971c-d5f8-4753-97ef-82e852ec81a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:08.145207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c971c-d5f8-4753-97ef-82e852ec81a8 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:08.148135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 831/1642] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:07 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:08.149173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03293530-8eb9-4f2c-b415-b840c44f19e7 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:08.170129 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03293530-8eb9-4f2c-b415-b840c44f19e7 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:08.405326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 812/1643] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:08.410428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740afb8-41c0-4c9b-9a6e-e4dd88b1898b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:08.410428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740afb8-41c0-4c9b-9a6e-e4dd88b1898b admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:08.410428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740afb8-41c0-4c9b-9a6e-e4dd88b1898b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:08.410428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740afb8-41c0-4c9b-9a6e-e4dd88b1898b admin admin] PATH_INFO: `/v3/projects/91ba5e94d5f141b793bc5f59793b9274/users/c76aa12ac4744957ad7292b005cb3df7/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:08.412084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740afb8-41c0-4c9b-9a6e-e4dd88b1898b admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ba5e94d5f141b793bc5f59793b9274, user_id=c76aa12ac4744957ad7292b005cb3df7, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:08.412084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740afb8-41c0-4c9b-9a6e-e4dd88b1898b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:08.412084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 832/1644] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:08 2022] PUT /identity/v3/projects/91ba5e94d5f141b793bc5f59793b9274/users/c76aa12ac4744957ad7292b005cb3df7/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:08.414312 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f591b-04cd-47b3-93f4-afedea3cc683 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:08.414312 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f591b-04cd-47b3-93f4-afedea3cc683 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:08.414312 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f591b-04cd-47b3-93f4-afedea3cc683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:08.416054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:08.417065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-202f591b-04cd-47b3-93f4-afedea3cc683 None None] MFA Rules not processed for user `c76aa12ac4744957ad7292b005cb3df7`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:08.417065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-202f591b-04cd-47b3-93f4-afedea3cc683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:08.421981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8295e9-4031-491d-be38-cc0af5613a7f None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:08.421981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8295e9-4031-491d-be38-cc0af5613a7f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:08.421981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8295e9-4031-491d-be38-cc0af5613a7f None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:08.421981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 833/1645] 10.222.0.69 () {54 vars in 912 bytes} [Thu Jul 28 06:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:28:08.421981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e747-7a56-4a49-8f10-ce04a7d52d6a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:08.421981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e747-7a56-4a49-8f10-ce04a7d52d6a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:08.423685 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e747-7a56-4a49-8f10-ce04a7d52d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:08.423867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:08.471088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e597e747-7a56-4a49-8f10-ce04a7d52d6a None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:08.471206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 813/1646] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:08.479626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e597e747-7a56-4a49-8f10-ce04a7d52d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:08.515951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f591b-04cd-47b3-93f4-afedea3cc683 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:08.606802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f591b-04cd-47b3-93f4-afedea3cc683 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:08.622334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1230633-69cc-4a7c-b9e4-1db54d31d874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:08.627469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1230633-69cc-4a7c-b9e4-1db54d31d874 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:08.627469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1230633-69cc-4a7c-b9e4-1db54d31d874 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:08.627469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1230633-69cc-4a7c-b9e4-1db54d31d874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:08.646291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 834/1647] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:28:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:08.650650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1230633-69cc-4a7c-b9e4-1db54d31d874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:08.652357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1230633-69cc-4a7c-b9e4-1db54d31d874 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:08.669374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 814/1648] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:08.678427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e747-7a56-4a49-8f10-ce04a7d52d6a None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:08.699233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e747-7a56-4a49-8f10-ce04a7d52d6a None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:08.704526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38559cac-a689-48c7-a2ab-8debd62999f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:08.705368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38559cac-a689-48c7-a2ab-8debd62999f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:08.705710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38559cac-a689-48c7-a2ab-8debd62999f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:08.706104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38559cac-a689-48c7-a2ab-8debd62999f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:08.718789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38559cac-a689-48c7-a2ab-8debd62999f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:08.720680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38559cac-a689-48c7-a2ab-8debd62999f3 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:08.735268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 835/1649] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:08.770401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1230633-69cc-4a7c-b9e4-1db54d31d874 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:08.780023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1230633-69cc-4a7c-b9e4-1db54d31d874 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:08.785095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80987d-8fab-4ec2-86d9-329fe2334bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:08.794792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80987d-8fab-4ec2-86d9-329fe2334bfb service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:08.794792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80987d-8fab-4ec2-86d9-329fe2334bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:08.794792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80987d-8fab-4ec2-86d9-329fe2334bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:08.909103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80987d-8fab-4ec2-86d9-329fe2334bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:08.917413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80987d-8fab-4ec2-86d9-329fe2334bfb service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:08.946901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 815/1650] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:09.062342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38559cac-a689-48c7-a2ab-8debd62999f3 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:09.062547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80987d-8fab-4ec2-86d9-329fe2334bfb service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:09.075097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80987d-8fab-4ec2-86d9-329fe2334bfb service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:09.084355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dec670-6e69-4618-b79b-c8c41807e580 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:09.085134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dec670-6e69-4618-b79b-c8c41807e580 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:09.085698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dec670-6e69-4618-b79b-c8c41807e580 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:09.086084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dec670-6e69-4618-b79b-c8c41807e580 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:09.196044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dec670-6e69-4618-b79b-c8c41807e580 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:09.197858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dec670-6e69-4618-b79b-c8c41807e580 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:09.223901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38559cac-a689-48c7-a2ab-8debd62999f3 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:09.224509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 816/1651] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:09.230086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:09.231356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:09.231736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:09.232086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] PATH_INFO: `/v3/users/5cc5c21a50d5452a8b9b9829f69ad0d2` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:09.233754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc5c21a50d5452a8b9b9829f69ad0d2)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:09.235283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: WARNING py.warnings [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:28:09.235283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: warnings.warn(msg) Jul 28 06:28:09.235283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: Jul 28 06:28:09.236555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:09.570456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc5c21a50d5452a8b9b9829f69ad0d2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:09.617151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc5c21a50d5452a8b9b9829f69ad0d2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:09.618418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc5c21a50d5452a8b9b9829f69ad0d2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:09.642958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc5c21a50d5452a8b9b9829f69ad0d2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:09.782992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc5c21a50d5452a8b9b9829f69ad0d2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:09.785747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:09.800856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 836/1652] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:09 2022] DELETE /identity/v3/users/5cc5c21a50d5452a8b9b9829f69ad0d2 => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:09.810125 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d287f4c-5cb9-4086-ba4f-bb54f8c479bb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:09.810639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d287f4c-5cb9-4086-ba4f-bb54f8c479bb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:09.810728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d287f4c-5cb9-4086-ba4f-bb54f8c479bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:09.844338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1d287f4c-5cb9-4086-ba4f-bb54f8c479bb None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:09.846075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1d287f4c-5cb9-4086-ba4f-bb54f8c479bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:09.942648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:10.011102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 817/1653] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:10.020706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:10.023774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:10.128285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbfb9b-7d29-45ac-a820-79a37bd85095 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:10.132677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b16741-a974-4131-b5f8-3f614c5fcfbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:10.134219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b16741-a974-4131-b5f8-3f614c5fcfbc service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:10.134626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b16741-a974-4131-b5f8-3f614c5fcfbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:10.135003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b16741-a974-4131-b5f8-3f614c5fcfbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:10.182042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b16741-a974-4131-b5f8-3f614c5fcfbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:10.182042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b16741-a974-4131-b5f8-3f614c5fcfbc service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:10.203826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 837/1654] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:11.507254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d287f4c-5cb9-4086-ba4f-bb54f8c479bb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:11.509683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1d287f4c-5cb9-4086-ba4f-bb54f8c479bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:11.557756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d287f4c-5cb9-4086-ba4f-bb54f8c479bb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:11.584131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:11.584876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:11.585164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:11.585482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] PATH_INFO: `/v3/projects/fc0bbee6f2334c599f7756b95c9d46dc` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:11.590864 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc0bbee6f2334c599f7756b95c9d46dc)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:11.595519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: WARNING py.warnings [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:28:11.595519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: warnings.warn(msg) Jul 28 06:28:11.595519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: Jul 28 06:28:11.601547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:11.832449 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:11.917423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc0bbee6f2334c599f7756b95c9d46dc', 'request_id': 'req-4bd2b0a1-622b-416e-b404-6f258fb8f637'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:11.928111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc0bbee6f2334c599f7756b95c9d46dc', 'request_id': 'req-4bd2b0a1-622b-416e-b404-6f258fb8f637'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:11.947139 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc0bbee6f2334c599f7756b95c9d46dc', 'request_id': 'req-4bd2b0a1-622b-416e-b404-6f258fb8f637'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:12.098416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] The token cache is being invalidate because project fc0bbee6f2334c599f7756b95c9d46dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:12.101137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:12.105461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4bd2b0a1-622b-416e-b404-6f258fb8f637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:12.115466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 818/1655] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:11 2022] DELETE /identity/v3/projects/fc0bbee6f2334c599f7756b95c9d46dc => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:12.121149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22463-3614-4c99-9fa5-995d7f171d1e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:12.121817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22463-3614-4c99-9fa5-995d7f171d1e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:12.122068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22463-3614-4c99-9fa5-995d7f171d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:12.173581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-0fc22463-3614-4c99-9fa5-995d7f171d1e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:12.175900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0fc22463-3614-4c99-9fa5-995d7f171d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:12.356172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:12.373092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 838/1656] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:12.385101 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df933f22-96f8-4b16-aac0-c143fe9541b8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:12.385422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df933f22-96f8-4b16-aac0-c143fe9541b8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:12.385788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df933f22-96f8-4b16-aac0-c143fe9541b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:12.428825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-df933f22-96f8-4b16-aac0-c143fe9541b8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:12.434770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-df933f22-96f8-4b16-aac0-c143fe9541b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:12.455118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 819/1657] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:12.459845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc22463-3614-4c99-9fa5-995d7f171d1e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:12.480871 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc22463-3614-4c99-9fa5-995d7f171d1e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:12.488004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38035d3d-2529-4847-ac5e-d6d3bf9be2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:12.489040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38035d3d-2529-4847-ac5e-d6d3bf9be2cf admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:12.489438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38035d3d-2529-4847-ac5e-d6d3bf9be2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:12.489801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38035d3d-2529-4847-ac5e-d6d3bf9be2cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:12.490658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38035d3d-2529-4847-ac5e-d6d3bf9be2cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:12.491213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38035d3d-2529-4847-ac5e-d6d3bf9be2cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:12.493138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38035d3d-2529-4847-ac5e-d6d3bf9be2cf admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:12.539984 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 839/1658] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:28:12.547003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df933f22-96f8-4b16-aac0-c143fe9541b8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:12.561015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df933f22-96f8-4b16-aac0-c143fe9541b8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:12.571037 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805a9a3-231d-4ad8-a82b-0c58cb60538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:12.572119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805a9a3-231d-4ad8-a82b-0c58cb60538b admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:12.572617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805a9a3-231d-4ad8-a82b-0c58cb60538b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:12.573559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805a9a3-231d-4ad8-a82b-0c58cb60538b admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:12.575119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805a9a3-231d-4ad8-a82b-0c58cb60538b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:12.579205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805a9a3-231d-4ad8-a82b-0c58cb60538b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:12.720331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 820/1659] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:12 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:28:12.724623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38035d3d-2529-4847-ac5e-d6d3bf9be2cf admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:12.740864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38035d3d-2529-4847-ac5e-d6d3bf9be2cf admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:12.829493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3026f-07df-47f7-be15-8a04317c328e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:12.833504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3026f-07df-47f7-be15-8a04317c328e admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:12.833504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3026f-07df-47f7-be15-8a04317c328e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:12.833504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3026f-07df-47f7-be15-8a04317c328e admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:12.833504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3026f-07df-47f7-be15-8a04317c328e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:12.843123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3026f-07df-47f7-be15-8a04317c328e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:12.978083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 840/1660] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:12 2022] POST /identity/v3/users => generated 286 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:12.981836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805a9a3-231d-4ad8-a82b-0c58cb60538b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:12.995424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805a9a3-231d-4ad8-a82b-0c58cb60538b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:13.001010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8c281-c245-4c1c-a094-abde7b6f01c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:13.002369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8c281-c245-4c1c-a094-abde7b6f01c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:13.003137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8c281-c245-4c1c-a094-abde7b6f01c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:13.003417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8c281-c245-4c1c-a094-abde7b6f01c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:13.008745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8c281-c245-4c1c-a094-abde7b6f01c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:13.018896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8c281-c245-4c1c-a094-abde7b6f01c7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:13.038443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 821/1661] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:13.043375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3026f-07df-47f7-be15-8a04317c328e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:13.070741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3026f-07df-47f7-be15-8a04317c328e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:13.080703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e7b6b-93f2-465d-84c6-aaee866bfa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:13.081836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e7b6b-93f2-465d-84c6-aaee866bfa91 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:13.082199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e7b6b-93f2-465d-84c6-aaee866bfa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:13.082590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e7b6b-93f2-465d-84c6-aaee866bfa91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:13.083733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e7b6b-93f2-465d-84c6-aaee866bfa91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:13.086809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e7b6b-93f2-465d-84c6-aaee866bfa91 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:13.104576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 841/1662] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:13.108823 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8c281-c245-4c1c-a094-abde7b6f01c7 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:13.128713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8c281-c245-4c1c-a094-abde7b6f01c7 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:13.143776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9a90a-a13e-4283-993e-8c6af50eb414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:13.143776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9a90a-a13e-4283-993e-8c6af50eb414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:13.143776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9a90a-a13e-4283-993e-8c6af50eb414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:13.143776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9a90a-a13e-4283-993e-8c6af50eb414 admin admin] PATH_INFO: `/v3/projects/4b695bc848964ccd8b7fd6d50e3682a2/users/ba6480efd53241f4960e99179fa9047b/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:13.161078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9a90a-a13e-4283-993e-8c6af50eb414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b695bc848964ccd8b7fd6d50e3682a2, user_id=ba6480efd53241f4960e99179fa9047b, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:13.163917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9a90a-a13e-4283-993e-8c6af50eb414 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:13.289005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 822/1663] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:13 2022] PUT /identity/v3/projects/4b695bc848964ccd8b7fd6d50e3682a2/users/ba6480efd53241f4960e99179fa9047b/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:13.290601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec68b0a-7c87-4536-aa66-8ea18651118c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:13.291479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec68b0a-7c87-4536-aa66-8ea18651118c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:13.291915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec68b0a-7c87-4536-aa66-8ea18651118c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:13.325985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-7ec68b0a-7c87-4536-aa66-8ea18651118c None None] MFA Rules not processed for user `ba6480efd53241f4960e99179fa9047b`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:13.327366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7ec68b0a-7c87-4536-aa66-8ea18651118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:13.413652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 842/1664] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:13.418164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9a90a-a13e-4283-993e-8c6af50eb414 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:13.539534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9a90a-a13e-4283-993e-8c6af50eb414 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:13.554872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb3f75-456c-40ff-bb53-dc14ea31be1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:13.557166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb3f75-456c-40ff-bb53-dc14ea31be1b admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:13.557514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb3f75-456c-40ff-bb53-dc14ea31be1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:13.560306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb3f75-456c-40ff-bb53-dc14ea31be1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:13.561163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb3f75-456c-40ff-bb53-dc14ea31be1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:13.570656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb3f75-456c-40ff-bb53-dc14ea31be1b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:13.792116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 823/1665] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:13 2022] POST /identity/v3/projects => generated 373 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:13.795191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec68b0a-7c87-4536-aa66-8ea18651118c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:13.813955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec68b0a-7c87-4536-aa66-8ea18651118c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:13.958258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19786e89-c532-4c84-97f2-d70ead581457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:13.959213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19786e89-c532-4c84-97f2-d70ead581457 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:13.959662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19786e89-c532-4c84-97f2-d70ead581457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:13.959959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19786e89-c532-4c84-97f2-d70ead581457 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:13.960658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19786e89-c532-4c84-97f2-d70ead581457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:13.970387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19786e89-c532-4c84-97f2-d70ead581457 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:14.116088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 843/1666] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:13 2022] POST /identity/v3/users => generated 293 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:14.119466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb3f75-456c-40ff-bb53-dc14ea31be1b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:14.135000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb3f75-456c-40ff-bb53-dc14ea31be1b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:14.151057 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352a812-5504-47cc-9db5-f7e0ae0d4980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:14.151701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352a812-5504-47cc-9db5-f7e0ae0d4980 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:14.152278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352a812-5504-47cc-9db5-f7e0ae0d4980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:14.160196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352a812-5504-47cc-9db5-f7e0ae0d4980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:14.164278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352a812-5504-47cc-9db5-f7e0ae0d4980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:14.166306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352a812-5504-47cc-9db5-f7e0ae0d4980 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:14.202108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19786e89-c532-4c84-97f2-d70ead581457 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:14.203417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 824/1667] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:14.220362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19786e89-c532-4c84-97f2-d70ead581457 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:14.229564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceb65e-a493-4c35-9b2b-cbeb67e9df4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:14.230197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceb65e-a493-4c35-9b2b-cbeb67e9df4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:14.230485 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceb65e-a493-4c35-9b2b-cbeb67e9df4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:14.231072 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceb65e-a493-4c35-9b2b-cbeb67e9df4c admin admin] PATH_INFO: `/v3/projects/23ab83a9cb464502a1e31e98cd2a0cf3/users/fbdbe18eaefd4dc2a34a91f94395ded7/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:14.254261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ceb65e-a493-4c35-9b2b-cbeb67e9df4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ab83a9cb464502a1e31e98cd2a0cf3, user_id=fbdbe18eaefd4dc2a34a91f94395ded7, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:14.257690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ceb65e-a493-4c35-9b2b-cbeb67e9df4c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:14.341750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352a812-5504-47cc-9db5-f7e0ae0d4980 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:14.342900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 844/1668] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:14 2022] PUT /identity/v3/projects/23ab83a9cb464502a1e31e98cd2a0cf3/users/fbdbe18eaefd4dc2a34a91f94395ded7/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:14.389734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352a812-5504-47cc-9db5-f7e0ae0d4980 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:14.479667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:14.492687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:14.589229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af56c-32ec-4ee7-80c2-558b326c804f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:14.606308 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af56c-32ec-4ee7-80c2-558b326c804f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:14.606595 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af56c-32ec-4ee7-80c2-558b326c804f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:14.607882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af56c-32ec-4ee7-80c2-558b326c804f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:14.613480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af56c-32ec-4ee7-80c2-558b326c804f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:14.624237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af56c-32ec-4ee7-80c2-558b326c804f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:14.650138 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 825/1669] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:14.654741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceb65e-a493-4c35-9b2b-cbeb67e9df4c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:14.672386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceb65e-a493-4c35-9b2b-cbeb67e9df4c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:14.689642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e97b8-6a94-46b0-aa98-2ec9263e22d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:14.691101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e97b8-6a94-46b0-aa98-2ec9263e22d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:14.691282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e97b8-6a94-46b0-aa98-2ec9263e22d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:14.691571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e97b8-6a94-46b0-aa98-2ec9263e22d6 admin admin] PATH_INFO: `/v3/projects/23ab83a9cb464502a1e31e98cd2a0cf3/users/fbdbe18eaefd4dc2a34a91f94395ded7/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:14.693933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e97b8-6a94-46b0-aa98-2ec9263e22d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ab83a9cb464502a1e31e98cd2a0cf3, user_id=fbdbe18eaefd4dc2a34a91f94395ded7, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:14.697564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e97b8-6a94-46b0-aa98-2ec9263e22d6 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:14.723840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 845/1670] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:14 2022] PUT /identity/v3/projects/23ab83a9cb464502a1e31e98cd2a0cf3/users/fbdbe18eaefd4dc2a34a91f94395ded7/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:28:14.733405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816aefc-ebfa-4c15-938e-895a9cbb6626 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:14.739766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816aefc-ebfa-4c15-938e-895a9cbb6626 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:14.739766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816aefc-ebfa-4c15-938e-895a9cbb6626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:14.820746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6816aefc-ebfa-4c15-938e-895a9cbb6626 None None] MFA Rules not processed for user `fbdbe18eaefd4dc2a34a91f94395ded7`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:14.826265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6816aefc-ebfa-4c15-938e-895a9cbb6626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:15.100436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 826/1671] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:15.129538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e97b8-6a94-46b0-aa98-2ec9263e22d6 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:15.129538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-be6e97b8-6a94-46b0-aa98-2ec9263e22d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:15.275424 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e97b8-6a94-46b0-aa98-2ec9263e22d6 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:15.289435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfdc19-d1c2-494a-b77b-4b6b50928908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:15.290376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfdc19-d1c2-494a-b77b-4b6b50928908 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:15.290674 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfdc19-d1c2-494a-b77b-4b6b50928908 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:15.290962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfdc19-d1c2-494a-b77b-4b6b50928908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:15.336884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfdc19-d1c2-494a-b77b-4b6b50928908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:15.338482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfdc19-d1c2-494a-b77b-4b6b50928908 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:15.379979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 846/1672] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:16.859400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816aefc-ebfa-4c15-938e-895a9cbb6626 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:16.987697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816aefc-ebfa-4c15-938e-895a9cbb6626 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:16.993185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:16.994072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:16.994375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:16.994660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] PATH_INFO: `/v3/users/ba6480efd53241f4960e99179fa9047b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:16.997431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba6480efd53241f4960e99179fa9047b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:16.999590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:17.275701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba6480efd53241f4960e99179fa9047b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:17.292484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:17.313494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba6480efd53241f4960e99179fa9047b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:17.341591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba6480efd53241f4960e99179fa9047b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:17.435659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba6480efd53241f4960e99179fa9047b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:17.436346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4da0c642-a3d4-44a5-8073-b800734478fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba6480efd53241f4960e99179fa9047b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:17.449917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 827/1673] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:16 2022] DELETE /identity/v3/users/ba6480efd53241f4960e99179fa9047b => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:17.453603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99f900-1e8f-48f1-be97-3c9c21d0f533 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:17.455607 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99f900-1e8f-48f1-be97-3c9c21d0f533 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:17.456829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99f900-1e8f-48f1-be97-3c9c21d0f533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:17.518164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2a99f900-1e8f-48f1-be97-3c9c21d0f533 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:17.520514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2a99f900-1e8f-48f1-be97-3c9c21d0f533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:17.593038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec137f6-7a26-49c7-b9d5-8e73b4ed734e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:17.593364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec137f6-7a26-49c7-b9d5-8e73b4ed734e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:17.593659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec137f6-7a26-49c7-b9d5-8e73b4ed734e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:17.663000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-dec137f6-7a26-49c7-b9d5-8e73b4ed734e None None] MFA Rules not processed for user `eda0cfe646954a9a8476b346ac9fed72`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:17.664273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dec137f6-7a26-49c7-b9d5-8e73b4ed734e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:17.726679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 847/1674] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:17.778459 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99f900-1e8f-48f1-be97-3c9c21d0f533 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:17.779387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2a99f900-1e8f-48f1-be97-3c9c21d0f533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:17.834296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 828/1675] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:17.838261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17fe0c-e309-4b14-ae07-821d78af5506 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:17.838462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17fe0c-e309-4b14-ae07-821d78af5506 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:17.838793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17fe0c-e309-4b14-ae07-821d78af5506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:17.880177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8a17fe0c-e309-4b14-ae07-821d78af5506 None None] MFA Rules not processed for user `eda0cfe646954a9a8476b346ac9fed72`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:17.884346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8a17fe0c-e309-4b14-ae07-821d78af5506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:17.908777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 829/1676] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:17.918044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17fe0c-e309-4b14-ae07-821d78af5506 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:17.922455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8a17fe0c-e309-4b14-ae07-821d78af5506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:17.941850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99f900-1e8f-48f1-be97-3c9c21d0f533 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:17.952710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44789e94-a031-486c-b5ec-6d7b2837578d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:17.953418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44789e94-a031-486c-b5ec-6d7b2837578d service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:17.953708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44789e94-a031-486c-b5ec-6d7b2837578d service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:17.954009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44789e94-a031-486c-b5ec-6d7b2837578d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:17.971613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:17.978898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44789e94-a031-486c-b5ec-6d7b2837578d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:17.982324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44789e94-a031-486c-b5ec-6d7b2837578d service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:18.009208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 848/1677] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:18.079539 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17fe0c-e309-4b14-ae07-821d78af5506 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:18.106287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2a09e-ff69-4260-a419-9851a14ae6fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:18.107048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2a09e-ff69-4260-a419-9851a14ae6fb service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:18.107406 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2a09e-ff69-4260-a419-9851a14ae6fb service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:18.107827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2a09e-ff69-4260-a419-9851a14ae6fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:18.143065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2a09e-ff69-4260-a419-9851a14ae6fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:18.145261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2a09e-ff69-4260-a419-9851a14ae6fb service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:18.183143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 830/1678] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:28:18.958774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44789e94-a031-486c-b5ec-6d7b2837578d service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:18.959369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-44789e94-a031-486c-b5ec-6d7b2837578d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:18.985424 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44789e94-a031-486c-b5ec-6d7b2837578d service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:19.002637 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:19.003561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:19.004615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:19.005575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] PATH_INFO: `/v3/users/a326cc97673246639b474bc742679b8d` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:19.008040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a326cc97673246639b474bc742679b8d)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:19.012994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:19.214887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2a09e-ff69-4260-a419-9851a14ae6fb service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:19.217022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0fc2a09e-ff69-4260-a419-9851a14ae6fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:19.255316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2a09e-ff69-4260-a419-9851a14ae6fb service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:19.273420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:19.275081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:19.275368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:19.275644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] PATH_INFO: `/v3/projects/4b695bc848964ccd8b7fd6d50e3682a2` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:19.276885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b695bc848964ccd8b7fd6d50e3682a2)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:19.281696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:19.392470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a326cc97673246639b474bc742679b8d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:19.436231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:19.470796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a326cc97673246639b474bc742679b8d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:19.473749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a326cc97673246639b474bc742679b8d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:19.476754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b695bc848964ccd8b7fd6d50e3682a2', 'request_id': 'req-edae3fd5-0b65-477c-9542-f1dd72426157'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:19.502751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a326cc97673246639b474bc742679b8d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:19.503999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b695bc848964ccd8b7fd6d50e3682a2', 'request_id': 'req-edae3fd5-0b65-477c-9542-f1dd72426157'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:19.520699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b695bc848964ccd8b7fd6d50e3682a2', 'request_id': 'req-edae3fd5-0b65-477c-9542-f1dd72426157'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:19.562262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d4e35f64-38b6-429f-b39e-f105eaeb3027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a326cc97673246639b474bc742679b8d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:19.575490 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 849/1679] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:18 2022] DELETE /identity/v3/users/a326cc97673246639b474bc742679b8d => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:19.578121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ba05c-46cf-4ca9-9025-0746d5ca4239 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:19.578457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ba05c-46cf-4ca9-9025-0746d5ca4239 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:19.580183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ba05c-46cf-4ca9-9025-0746d5ca4239 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:19.583309 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 850/1680] 10.222.0.69 () {54 vars in 912 bytes} [Thu Jul 28 06:28:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:19.584764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f15a46-19be-4932-9afe-a7fa2b2cbe30 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:19.585327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f15a46-19be-4932-9afe-a7fa2b2cbe30 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:19.585841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f15a46-19be-4932-9afe-a7fa2b2cbe30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:19.632378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] The token cache is being invalidate because project 4b695bc848964ccd8b7fd6d50e3682a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:19.636813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:19.636813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edae3fd5-0b65-477c-9542-f1dd72426157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:19.637185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-c8f15a46-19be-4932-9afe-a7fa2b2cbe30 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:19.637924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 831/1681] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:19 2022] DELETE /identity/v3/projects/4b695bc848964ccd8b7fd6d50e3682a2 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:19.640745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8200f7-e006-4f66-9c6d-2b786fdc38c3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:19.641242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8200f7-e006-4f66-9c6d-2b786fdc38c3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:19.641988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8200f7-e006-4f66-9c6d-2b786fdc38c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:19.642831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c8f15a46-19be-4932-9afe-a7fa2b2cbe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:19.741380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 851/1682] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:19.742687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f15a46-19be-4932-9afe-a7fa2b2cbe30 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:19.744274 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c8f15a46-19be-4932-9afe-a7fa2b2cbe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:19.767873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f15a46-19be-4932-9afe-a7fa2b2cbe30 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:19.774240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5b8200f7-e006-4f66-9c6d-2b786fdc38c3 None None] MFA Rules not processed for user `05574eace38f4b6d945c7b1927fdadd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:19.775887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5b8200f7-e006-4f66-9c6d-2b786fdc38c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:19.782595 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:19.786735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:19.791819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:19.791819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] PATH_INFO: `/v3/users/fbdbe18eaefd4dc2a34a91f94395ded7` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:19.792873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbdbe18eaefd4dc2a34a91f94395ded7)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:19.802373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:19.887276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 832/1683] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:28:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:19.917903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8200f7-e006-4f66-9c6d-2b786fdc38c3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:19.921190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5b8200f7-e006-4f66-9c6d-2b786fdc38c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:19.941195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8200f7-e006-4f66-9c6d-2b786fdc38c3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:19.946388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c7105-929f-480f-a116-4d26f9884830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:19.947416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c7105-929f-480f-a116-4d26f9884830 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:19.947821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c7105-929f-480f-a116-4d26f9884830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:19.948293 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c7105-929f-480f-a116-4d26f9884830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:19.965685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c7105-929f-480f-a116-4d26f9884830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:19.967863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c7105-929f-480f-a116-4d26f9884830 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:20.026609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 833/1684] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:28:20.031764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c7105-929f-480f-a116-4d26f9884830 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:20.059442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c7105-929f-480f-a116-4d26f9884830 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:20.070669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814bcea-673f-4da3-95bb-7fdb36d47444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:20.074362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814bcea-673f-4da3-95bb-7fdb36d47444 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:20.074720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814bcea-673f-4da3-95bb-7fdb36d47444 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:20.075286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814bcea-673f-4da3-95bb-7fdb36d47444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:20.102617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814bcea-673f-4da3-95bb-7fdb36d47444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:20.104388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814bcea-673f-4da3-95bb-7fdb36d47444 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:20.133751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbdbe18eaefd4dc2a34a91f94395ded7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:20.150658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 834/1685] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:20.151692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:20.162405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbdbe18eaefd4dc2a34a91f94395ded7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:20.163680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbdbe18eaefd4dc2a34a91f94395ded7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:20.181268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbdbe18eaefd4dc2a34a91f94395ded7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:20.258588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8516727-6fe5-4383-8b30-9d74dec42dde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbdbe18eaefd4dc2a34a91f94395ded7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:20.270410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 852/1686] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:19 2022] DELETE /identity/v3/users/fbdbe18eaefd4dc2a34a91f94395ded7 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:21.437651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cd153-72e8-4ee4-90f0-add5e78db1f8 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:21.439512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cd153-72e8-4ee4-90f0-add5e78db1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:21.439512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cd153-72e8-4ee4-90f0-add5e78db1f8 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:21.441082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 835/1687] 10.222.0.69 () {54 vars in 961 bytes} [Thu Jul 28 06:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:21.448303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b0eb-671e-4a25-bc4e-4b81156da26c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:21.448772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b0eb-671e-4a25-bc4e-4b81156da26c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:21.448772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b0eb-671e-4a25-bc4e-4b81156da26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:21.512063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-c3c0b0eb-671e-4a25-bc4e-4b81156da26c None None] MFA Rules not processed for user `244a8787df7641f2bac9b51213ddb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:21.513366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c3c0b0eb-671e-4a25-bc4e-4b81156da26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:21.666183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cd153-72e8-4ee4-90f0-add5e78db1f8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:21.668750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-681cd153-72e8-4ee4-90f0-add5e78db1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:21.859644 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 853/1688] 10.222.0.69 () {58 vars in 1058 bytes} [Thu Jul 28 06:28:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:21.861051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56b355-b915-47ef-af94-f3ef6b119420 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:21.862524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56b355-b915-47ef-af94-f3ef6b119420 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:21.863440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56b355-b915-47ef-af94-f3ef6b119420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:21.919126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cd153-72e8-4ee4-90f0-add5e78db1f8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:21.924055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-7f56b355-b915-47ef-af94-f3ef6b119420 None None] MFA Rules not processed for user `86a5553819d949ffbc64b7a388942baf`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:21.926254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7f56b355-b915-47ef-af94-f3ef6b119420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:21.940644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:21.947828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:21.948347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:21.948677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] PATH_INFO: `/v3/projects/f6847d20741447f1b71a41e08fe90876` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:21.951173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6847d20741447f1b71a41e08fe90876)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:21.955429 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:22.038191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 854/1689] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:22.040043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56b355-b915-47ef-af94-f3ef6b119420 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:22.062281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56b355-b915-47ef-af94-f3ef6b119420 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:22.080297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181ffe5-1979-4431-a6a9-2a0216897bdf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:22.080297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181ffe5-1979-4431-a6a9-2a0216897bdf service placement] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:22.080297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181ffe5-1979-4431-a6a9-2a0216897bdf service placement] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:22.080297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181ffe5-1979-4431-a6a9-2a0216897bdf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:22.086798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b181ffe5-1979-4431-a6a9-2a0216897bdf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:22.106320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181ffe5-1979-4431-a6a9-2a0216897bdf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:22.108124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181ffe5-1979-4431-a6a9-2a0216897bdf service placement] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:22.117843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:22.120811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 855/1690] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:22.122230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe6bf0-beb1-4109-b54f-5777819781d5 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:22.122588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe6bf0-beb1-4109-b54f-5777819781d5 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:22.122999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe6bf0-beb1-4109-b54f-5777819781d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:22.162700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-78fe6bf0-beb1-4109-b54f-5777819781d5 None None] MFA Rules not processed for user `86a5553819d949ffbc64b7a388942baf`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:22.164303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-78fe6bf0-beb1-4109-b54f-5777819781d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:22.181055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6847d20741447f1b71a41e08fe90876', 'request_id': 'req-5a009820-ac39-41fc-bb8a-a5994d47c6f8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:22.205112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6847d20741447f1b71a41e08fe90876', 'request_id': 'req-5a009820-ac39-41fc-bb8a-a5994d47c6f8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:22.225663 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6847d20741447f1b71a41e08fe90876', 'request_id': 'req-5a009820-ac39-41fc-bb8a-a5994d47c6f8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:22.267585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:22.304312 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 856/1691] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:22.310667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe6bf0-beb1-4109-b54f-5777819781d5 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:22.317859 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-78fe6bf0-beb1-4109-b54f-5777819781d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:22.357868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] The token cache is being invalidate because project f6847d20741447f1b71a41e08fe90876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:22.358225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:22.359811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:22.362623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 836/1692] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:21 2022] DELETE /identity/v3/projects/f6847d20741447f1b71a41e08fe90876 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:22.370538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:22.374076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:22.496947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe6bf0-beb1-4109-b54f-5777819781d5 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:22.509973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852c52c-ce14-40ab-985b-2fd93594928c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:22.511475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852c52c-ce14-40ab-985b-2fd93594928c service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:22.513847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852c52c-ce14-40ab-985b-2fd93594928c service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:22.514354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852c52c-ce14-40ab-985b-2fd93594928c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:22.522066 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d852c52c-ce14-40ab-985b-2fd93594928c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:22.550102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a009820-ac39-41fc-bb8a-a5994d47c6f8 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:22.555185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852c52c-ce14-40ab-985b-2fd93594928c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:22.560595 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852c52c-ce14-40ab-985b-2fd93594928c service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:22.561532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:22.562205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:22.565652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:22.565945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] PATH_INFO: `/v3/users/5a9e2a9962de4a7897ac0e3abde497d2` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:22.568912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a9e2a9962de4a7897ac0e3abde497d2)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:22.570783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:22.584266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 857/1693] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:28:22.850127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a9e2a9962de4a7897ac0e3abde497d2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:22.884547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a9e2a9962de4a7897ac0e3abde497d2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:22.909658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a9e2a9962de4a7897ac0e3abde497d2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:23.059627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a9e2a9962de4a7897ac0e3abde497d2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:23.070575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a9e2a9962de4a7897ac0e3abde497d2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:23.085340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 837/1694] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:22 2022] DELETE /identity/v3/users/5a9e2a9962de4a7897ac0e3abde497d2 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:24.473098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852c52c-ce14-40ab-985b-2fd93594928c service nova] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:24.474603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d852c52c-ce14-40ab-985b-2fd93594928c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:24.504514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852c52c-ce14-40ab-985b-2fd93594928c service nova] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:24.678947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:24.679647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:24.679936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:24.680567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] PATH_INFO: `/v3/projects/23ab83a9cb464502a1e31e98cd2a0cf3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:24.686322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ab83a9cb464502a1e31e98cd2a0cf3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:24.688503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:24.873645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ab83a9cb464502a1e31e98cd2a0cf3', 'request_id': 'req-d8ea3e00-1420-4d9c-8268-8cf388371aca'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:24.885049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ab83a9cb464502a1e31e98cd2a0cf3', 'request_id': 'req-d8ea3e00-1420-4d9c-8268-8cf388371aca'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:24.942575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:24.944683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:24.955245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ab83a9cb464502a1e31e98cd2a0cf3', 'request_id': 'req-d8ea3e00-1420-4d9c-8268-8cf388371aca'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:24.981835 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] The token cache is being invalidate because project 23ab83a9cb464502a1e31e98cd2a0cf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:24.982359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:24.984317 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d8ea3e00-1420-4d9c-8268-8cf388371aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:24.985498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73903aea-eddb-4f5a-bfb7-34c99a392879 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:24.989982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 858/1695] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:24 2022] DELETE /identity/v3/projects/23ab83a9cb464502a1e31e98cd2a0cf3 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:24.996625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634c493-238d-4d9a-8593-96148c002e6a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:24.999749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634c493-238d-4d9a-8593-96148c002e6a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:25.002975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634c493-238d-4d9a-8593-96148c002e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:25.056664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4634c493-238d-4d9a-8593-96148c002e6a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:25.059812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4634c493-238d-4d9a-8593-96148c002e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:25.075995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:25.089117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:25.155083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:25.156138 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:25.157269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:25.158586 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] PATH_INFO: `/v3/projects/6bcab5b6799b45079b12f9f4fd4f6e4e` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:25.160397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bcab5b6799b45079b12f9f4fd4f6e4e)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:25.170949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:25.275030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 859/1696] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:25.276710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a59b9-e7e7-4bc3-8703-947a54d414e9 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:25.277145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a59b9-e7e7-4bc3-8703-947a54d414e9 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:25.278205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a59b9-e7e7-4bc3-8703-947a54d414e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:25.323803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2c5a59b9-e7e7-4bc3-8703-947a54d414e9 None None] MFA Rules not processed for user `61ad8186d91848bc8be42e3be51e9a75`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:25.325780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2c5a59b9-e7e7-4bc3-8703-947a54d414e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:25.456435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 860/1697] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:25.456435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c659573-c69e-493a-8122-89a2aba9ce70 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:25.456435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c659573-c69e-493a-8122-89a2aba9ce70 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:25.456435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c659573-c69e-493a-8122-89a2aba9ce70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:25.480900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bcab5b6799b45079b12f9f4fd4f6e4e', 'request_id': 'req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:25.483895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4c659573-c69e-493a-8122-89a2aba9ce70 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:25.483895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4c659573-c69e-493a-8122-89a2aba9ce70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:25.498699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bcab5b6799b45079b12f9f4fd4f6e4e', 'request_id': 'req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:25.512734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bcab5b6799b45079b12f9f4fd4f6e4e', 'request_id': 'req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:25.596686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] The token cache is being invalidate because project 6bcab5b6799b45079b12f9f4fd4f6e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:25.597032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:25.598593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:25.600900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 838/1698] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:24 2022] DELETE /identity/v3/projects/6bcab5b6799b45079b12f9f4fd4f6e4e => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:25.601830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:25.609476 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:25.611277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 861/1699] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:25.613029 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a17452-0910-4b7e-b48e-ac801018e65f None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:25.615016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a17452-0910-4b7e-b48e-ac801018e65f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:25.617627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a17452-0910-4b7e-b48e-ac801018e65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:25.688678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b2a17452-0910-4b7e-b48e-ac801018e65f None None] MFA Rules not processed for user `86a5553819d949ffbc64b7a388942baf`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:25.693664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b2a17452-0910-4b7e-b48e-ac801018e65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:25.880610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4fe2b-ecf5-4dbc-9140-3f9cb7d2c14b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:25.909271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd2923-ce54-481b-bf65-d50cf5fc8352 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:25.909271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd2923-ce54-481b-bf65-d50cf5fc8352 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:25.909584 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd2923-ce54-481b-bf65-d50cf5fc8352 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:25.909881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd2923-ce54-481b-bf65-d50cf5fc8352 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:25.921723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3ddd2923-ce54-481b-bf65-d50cf5fc8352 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:26.002647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 862/1700] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:26.006294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a17452-0910-4b7e-b48e-ac801018e65f None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:26.008964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b2a17452-0910-4b7e-b48e-ac801018e65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:26.063876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a17452-0910-4b7e-b48e-ac801018e65f None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:26.085554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:26.086526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:26.087171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:26.087654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] PATH_INFO: `/v3/users/c76aa12ac4744957ad7292b005cb3df7` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:26.089510 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c76aa12ac4744957ad7292b005cb3df7)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:26.094090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:26.163182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd2923-ce54-481b-bf65-d50cf5fc8352 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:26.164844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd2923-ce54-481b-bf65-d50cf5fc8352 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:26.201555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 839/1701] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:26.202723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd2923-ce54-481b-bf65-d50cf5fc8352 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:26.219530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3ddd2923-ce54-481b-bf65-d50cf5fc8352 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:26.275387 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd2923-ce54-481b-bf65-d50cf5fc8352 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:26.315685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8f3f4-7afd-40a1-86d9-8222732395b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:26.317158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8f3f4-7afd-40a1-86d9-8222732395b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:26.323294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8f3f4-7afd-40a1-86d9-8222732395b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:26.323813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8f3f4-7afd-40a1-86d9-8222732395b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:26.325819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8f3f4-7afd-40a1-86d9-8222732395b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:26.337975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8f3f4-7afd-40a1-86d9-8222732395b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:26.339665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8f3f4-7afd-40a1-86d9-8222732395b4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:26.412690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c76aa12ac4744957ad7292b005cb3df7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:26.424671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 840/1702] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:26.425709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8f3f4-7afd-40a1-86d9-8222732395b4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:26.453829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8f3f4-7afd-40a1-86d9-8222732395b4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:26.470703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d6824-dac7-42e7-9ab2-9f60b6a0bc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:26.474616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d6824-dac7-42e7-9ab2-9f60b6a0bc04 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:26.474912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d6824-dac7-42e7-9ab2-9f60b6a0bc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:26.475203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d6824-dac7-42e7-9ab2-9f60b6a0bc04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:26.475906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d6824-dac7-42e7-9ab2-9f60b6a0bc04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:26.478545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c76aa12ac4744957ad7292b005cb3df7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:26.482077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c76aa12ac4744957ad7292b005cb3df7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:26.505758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d6824-dac7-42e7-9ab2-9f60b6a0bc04 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:26.521983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c76aa12ac4744957ad7292b005cb3df7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:26.623959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c76aa12ac4744957ad7292b005cb3df7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:26.633262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 863/1703] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:26 2022] DELETE /identity/v3/users/c76aa12ac4744957ad7292b005cb3df7 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:26.660863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 841/1704] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:26 2022] POST /identity/v3/projects => generated 363 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:26.661476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:26.663081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:26.887228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dc7ba-15f6-4863-a210-32fd95c3904f admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:26.928182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:27.104224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:27.122868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa86d2-5ef2-4fac-8812-dd7f307cde01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:27.127749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa86d2-5ef2-4fac-8812-dd7f307cde01 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:27.128038 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa86d2-5ef2-4fac-8812-dd7f307cde01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:27.130800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa86d2-5ef2-4fac-8812-dd7f307cde01 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:27.131798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa86d2-5ef2-4fac-8812-dd7f307cde01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:27.143961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa86d2-5ef2-4fac-8812-dd7f307cde01 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:27.324412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 864/1705] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:26 2022] POST /identity/v3/users => generated 282 bytes in 663 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:28:27.328382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d6824-dac7-42e7-9ab2-9f60b6a0bc04 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:27.355842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6ac97-97d5-4f8b-82d6-bfe32d5e93c4 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:27.358893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6ac97-97d5-4f8b-82d6-bfe32d5e93c4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:27.358893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6ac97-97d5-4f8b-82d6-bfe32d5e93c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:27.366633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d6824-dac7-42e7-9ab2-9f60b6a0bc04 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:27.421532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb86f7-94b2-416b-b274-58ba65230041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:27.426076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb86f7-94b2-416b-b274-58ba65230041 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:27.426370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb86f7-94b2-416b-b274-58ba65230041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:27.426659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb86f7-94b2-416b-b274-58ba65230041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:27.427401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb86f7-94b2-416b-b274-58ba65230041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:27.440801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb86f7-94b2-416b-b274-58ba65230041 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:27.502267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 842/1706] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:27.507108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf5628-9141-44de-8d08-13c22c1aa272 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:27.513890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf5628-9141-44de-8d08-13c22c1aa272 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:27.516274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf5628-9141-44de-8d08-13c22c1aa272 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:27.525065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 843/1707] 10.222.0.69 () {54 vars in 912 bytes} [Thu Jul 28 06:28:27 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:27.527005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf5628-9141-44de-8d08-13c22c1aa272 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:27.540573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e0f6ac97-97d5-4f8b-82d6-bfe32d5e93c4 None None] MFA Rules not processed for user `6d4528589ffa478689e905d47e4c37ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:27.549231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e0f6ac97-97d5-4f8b-82d6-bfe32d5e93c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:27.573717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf5628-9141-44de-8d08-13c22c1aa272 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:27.591000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619a954-aa5d-41a2-956f-76c4b7621e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:27.593299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619a954-aa5d-41a2-956f-76c4b7621e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:27.594598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619a954-aa5d-41a2-956f-76c4b7621e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:27.596571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619a954-aa5d-41a2-956f-76c4b7621e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:27.598690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619a954-aa5d-41a2-956f-76c4b7621e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:27.603960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619a954-aa5d-41a2-956f-76c4b7621e7d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:27.619569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:27.647321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 844/1708] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:27.649206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e980-da57-4775-83b4-42ceb5f78661 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:27.649847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e980-da57-4775-83b4-42ceb5f78661 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:27.659809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e980-da57-4775-83b4-42ceb5f78661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:27.716602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c5a6e980-da57-4775-83b4-42ceb5f78661 None None] MFA Rules not processed for user `05574eace38f4b6d945c7b1927fdadd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:27.718618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c5a6e980-da57-4775-83b4-42ceb5f78661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:27.793473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 865/1709] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:27.794463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6ac97-97d5-4f8b-82d6-bfe32d5e93c4 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:27.826709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 845/1710] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:28:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:27.830802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb0980-3b99-4b02-8f11-7da290fdbf30 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:27.831945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb0980-3b99-4b02-8f11-7da290fdbf30 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:27.832056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6ac97-97d5-4f8b-82d6-bfe32d5e93c4 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:27.833139 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb0980-3b99-4b02-8f11-7da290fdbf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:27.845972 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac731ae9-e599-4861-9aec-4c1608ed6837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:27.846696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac731ae9-e599-4861-9aec-4c1608ed6837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:27.846987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac731ae9-e599-4861-9aec-4c1608ed6837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:27.847282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac731ae9-e599-4861-9aec-4c1608ed6837 admin admin] PATH_INFO: `/v3/projects/6ec77a76d1da455b9a2483a60421e699/users/1144f2ad9639415f90f2dd71a6d91158/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:27.906646 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac731ae9-e599-4861-9aec-4c1608ed6837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec77a76d1da455b9a2483a60421e699, user_id=1144f2ad9639415f90f2dd71a6d91158, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:27.913219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac731ae9-e599-4861-9aec-4c1608ed6837 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:28.003719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4feb0980-3b99-4b02-8f11-7da290fdbf30 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:28.004926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4feb0980-3b99-4b02-8f11-7da290fdbf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:28.045658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 866/1711] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:27 2022] PUT /identity/v3/projects/6ec77a76d1da455b9a2483a60421e699/users/1144f2ad9639415f90f2dd71a6d91158/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:28.047335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac731ae9-e599-4861-9aec-4c1608ed6837 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:28.050330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ac731ae9-e599-4861-9aec-4c1608ed6837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:28.138930 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 846/1712] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:28:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:28.145202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb0980-3b99-4b02-8f11-7da290fdbf30 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:28.147881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4feb0980-3b99-4b02-8f11-7da290fdbf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:28.212073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac731ae9-e599-4861-9aec-4c1608ed6837 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:28.228957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d06df5-e020-4255-b9d6-bc9516f05d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:28.229542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d06df5-e020-4255-b9d6-bc9516f05d4b service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:28.229927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d06df5-e020-4255-b9d6-bc9516f05d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:28.230364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d06df5-e020-4255-b9d6-bc9516f05d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:28.259065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d06df5-e020-4255-b9d6-bc9516f05d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:28.260688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d06df5-e020-4255-b9d6-bc9516f05d4b service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:28.286864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 867/1713] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:28.298589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974216b-00bf-44e1-b099-3b2266eca595 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:28.298589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974216b-00bf-44e1-b099-3b2266eca595 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:28.301375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974216b-00bf-44e1-b099-3b2266eca595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:28.444139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-9974216b-00bf-44e1-b099-3b2266eca595 None None] MFA Rules not processed for user `1144f2ad9639415f90f2dd71a6d91158`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:28.458123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9974216b-00bf-44e1-b099-3b2266eca595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:28.461685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb0980-3b99-4b02-8f11-7da290fdbf30 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:28.646115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 868/1714] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:28 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:28.651367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974216b-00bf-44e1-b099-3b2266eca595 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:28.707137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8534e5-d85e-4a64-bdbe-1c44182c985d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:28.707847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8534e5-d85e-4a64-bdbe-1c44182c985d service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:28.708309 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8534e5-d85e-4a64-bdbe-1c44182c985d service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:28.708743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8534e5-d85e-4a64-bdbe-1c44182c985d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:28.831738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974216b-00bf-44e1-b099-3b2266eca595 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:28.837320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0837e-e7f0-4ea3-a726-4d31f52a979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:28.841325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8534e5-d85e-4a64-bdbe-1c44182c985d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:28.843102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8534e5-d85e-4a64-bdbe-1c44182c985d service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:28.854712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0837e-e7f0-4ea3-a726-4d31f52a979b admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:28.855006 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0837e-e7f0-4ea3-a726-4d31f52a979b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:28.855309 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0837e-e7f0-4ea3-a726-4d31f52a979b admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:28.856289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0837e-e7f0-4ea3-a726-4d31f52a979b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:28.859695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 847/1715] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:28.860553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8534e5-d85e-4a64-bdbe-1c44182c985d service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:28.864223 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0837e-e7f0-4ea3-a726-4d31f52a979b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:28.868913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fb8534e5-d85e-4a64-bdbe-1c44182c985d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:28.927646 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:29.066125 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8534e5-d85e-4a64-bdbe-1c44182c985d service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:29.073067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 869/1716] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:28 2022] POST /identity/v3/projects => generated 363 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:29.103459 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0837e-e7f0-4ea3-a726-4d31f52a979b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:29.206250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1925597-845d-4cc1-87b1-d7fac677dbcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:29.206822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1925597-845d-4cc1-87b1-d7fac677dbcb service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:29.207113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1925597-845d-4cc1-87b1-d7fac677dbcb service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:29.207397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1925597-845d-4cc1-87b1-d7fac677dbcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:29.279027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0837e-e7f0-4ea3-a726-4d31f52a979b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:29.293662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cab18-88e6-4697-8405-d3fe0fd6053e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:29.294513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cab18-88e6-4697-8405-d3fe0fd6053e admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:29.294810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cab18-88e6-4697-8405-d3fe0fd6053e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:29.295096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cab18-88e6-4697-8405-d3fe0fd6053e admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:29.295938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cab18-88e6-4697-8405-d3fe0fd6053e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:29.299459 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cab18-88e6-4697-8405-d3fe0fd6053e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:29.317282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:29.386254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1925597-845d-4cc1-87b1-d7fac677dbcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:29.389112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1925597-845d-4cc1-87b1-d7fac677dbcb service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:29.400565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 848/1717] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:28:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:28:29.450001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 870/1718] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:29 2022] POST /identity/v3/users => generated 288 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:29.460919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1925597-845d-4cc1-87b1-d7fac677dbcb service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:29.468996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1925597-845d-4cc1-87b1-d7fac677dbcb service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:29.475466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599c00a-3bf0-4a69-939f-ce8fd2a91c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:29.476226 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599c00a-3bf0-4a69-939f-ce8fd2a91c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:29.476558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599c00a-3bf0-4a69-939f-ce8fd2a91c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:29.476844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599c00a-3bf0-4a69-939f-ce8fd2a91c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:29.477842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599c00a-3bf0-4a69-939f-ce8fd2a91c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:29.480225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599c00a-3bf0-4a69-939f-ce8fd2a91c84 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:29.503136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 849/1719] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:29.504885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cab18-88e6-4697-8405-d3fe0fd6053e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:29.519845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cab18-88e6-4697-8405-d3fe0fd6053e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:29.537932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff3b68-d060-4469-a435-5c6a0e960057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:29.540255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff3b68-d060-4469-a435-5c6a0e960057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:29.541499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff3b68-d060-4469-a435-5c6a0e960057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:29.543222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff3b68-d060-4469-a435-5c6a0e960057 admin admin] PATH_INFO: `/v3/projects/8e03f432f7bf43a3a418138e69e96f0c/users/cf724782aa204022a5f35fae4a9aa684/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:29.604517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff3b68-d060-4469-a435-5c6a0e960057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e03f432f7bf43a3a418138e69e96f0c, user_id=cf724782aa204022a5f35fae4a9aa684, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:29.614110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff3b68-d060-4469-a435-5c6a0e960057 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:29.724036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 871/1720] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:29 2022] PUT /identity/v3/projects/8e03f432f7bf43a3a418138e69e96f0c/users/cf724782aa204022a5f35fae4a9aa684/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:29.727859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599c00a-3bf0-4a69-939f-ce8fd2a91c84 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:29.747627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599c00a-3bf0-4a69-939f-ce8fd2a91c84 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:29.978236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d25669-f423-42dd-9895-c674d29801d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:29.980065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d25669-f423-42dd-9895-c674d29801d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:29.980065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d25669-f423-42dd-9895-c674d29801d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:29.980065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d25669-f423-42dd-9895-c674d29801d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:29.982291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d25669-f423-42dd-9895-c674d29801d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:29.984648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d25669-f423-42dd-9895-c674d29801d9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:30.002382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 850/1721] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:30.003075 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff3b68-d060-4469-a435-5c6a0e960057 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:30.019046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff3b68-d060-4469-a435-5c6a0e960057 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:30.049089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f6acf-ab11-465e-9b9b-46a57a4f8197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:30.051764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f6acf-ab11-465e-9b9b-46a57a4f8197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:30.057789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f6acf-ab11-465e-9b9b-46a57a4f8197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:30.066692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f6acf-ab11-465e-9b9b-46a57a4f8197 admin admin] PATH_INFO: `/v3/projects/8e03f432f7bf43a3a418138e69e96f0c/users/cf724782aa204022a5f35fae4a9aa684/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:30.066692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f6acf-ab11-465e-9b9b-46a57a4f8197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e03f432f7bf43a3a418138e69e96f0c, user_id=cf724782aa204022a5f35fae4a9aa684, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:30.075635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f6acf-ab11-465e-9b9b-46a57a4f8197 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:30.088663 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:30.124497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 872/1722] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:30 2022] PUT /identity/v3/projects/8e03f432f7bf43a3a418138e69e96f0c/users/cf724782aa204022a5f35fae4a9aa684/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:28:30.136477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85563c7f-2b30-40ce-b944-39f957f78767 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:30.136763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85563c7f-2b30-40ce-b944-39f957f78767 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:30.137198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85563c7f-2b30-40ce-b944-39f957f78767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:30.276770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-85563c7f-2b30-40ce-b944-39f957f78767 None None] MFA Rules not processed for user `cf724782aa204022a5f35fae4a9aa684`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:30.291322 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-85563c7f-2b30-40ce-b944-39f957f78767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:30.596094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:30.687379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 851/1723] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:30 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 553 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:30.707868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f6acf-ab11-465e-9b9b-46a57a4f8197 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:30.710016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-207f6acf-ab11-465e-9b9b-46a57a4f8197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:30.763421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:30.859108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f6acf-ab11-465e-9b9b-46a57a4f8197 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:30.871876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9215c-1892-4acf-95ce-27415b6f2d31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:30.873352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9215c-1892-4acf-95ce-27415b6f2d31 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:30.876683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9215c-1892-4acf-95ce-27415b6f2d31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:30.880116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9215c-1892-4acf-95ce-27415b6f2d31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:30.899024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9215c-1892-4acf-95ce-27415b6f2d31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:30.905883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9215c-1892-4acf-95ce-27415b6f2d31 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:30.972781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 873/1724] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:30 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:31.691661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e84209-2654-47b2-88e1-326debaba518 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:31.692867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e84209-2654-47b2-88e1-326debaba518 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:31.693457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e84209-2654-47b2-88e1-326debaba518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:31.742687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-04e84209-2654-47b2-88e1-326debaba518 None None] MFA Rules not processed for user `0d268e37c8b64bcc9261a80bb0461edc`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:31.746596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-04e84209-2654-47b2-88e1-326debaba518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:31.755175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9215c-1892-4acf-95ce-27415b6f2d31 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:31.776579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9215c-1892-4acf-95ce-27415b6f2d31 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:31.802324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c6a72-342e-45e1-a697-086b1643d2ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:31.802324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c6a72-342e-45e1-a697-086b1643d2ae service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:31.802324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c6a72-342e-45e1-a697-086b1643d2ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:31.802610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c6a72-342e-45e1-a697-086b1643d2ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:32.113404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 852/1725] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:32.138630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e84209-2654-47b2-88e1-326debaba518 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:32.139801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c6a72-342e-45e1-a697-086b1643d2ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:32.141444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c6a72-342e-45e1-a697-086b1643d2ae service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:32.146479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-04e84209-2654-47b2-88e1-326debaba518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:32.179905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 874/1726] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:32.180992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c6a72-342e-45e1-a697-086b1643d2ae service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:32.476905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e84209-2654-47b2-88e1-326debaba518 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:32.506106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:32.506846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:32.507132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:32.507412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] PATH_INFO: `/v3/projects/91ba5e94d5f141b793bc5f59793b9274` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:32.509388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ba5e94d5f141b793bc5f59793b9274)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:32.513885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:32.530608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c6a72-342e-45e1-a697-086b1643d2ae service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:32.552699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8b401-67ce-4b7e-9520-e474f91dfdda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:32.553387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8b401-67ce-4b7e-9520-e474f91dfdda service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:32.553667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8b401-67ce-4b7e-9520-e474f91dfdda service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:32.553933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8b401-67ce-4b7e-9520-e474f91dfdda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:32.575055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8b401-67ce-4b7e-9520-e474f91dfdda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:32.577985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8b401-67ce-4b7e-9520-e474f91dfdda service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:32.612641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 875/1727] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:32.667929 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d773d-0dab-4068-94e0-434a0054ce40 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:32.668656 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d773d-0dab-4068-94e0-434a0054ce40 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:32.669082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d773d-0dab-4068-94e0-434a0054ce40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:32.694330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:32.742616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ba5e94d5f141b793bc5f59793b9274', 'request_id': 'req-0feb8499-ce08-4a15-9710-2df13d772cb6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:32.754261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-da0d773d-0dab-4068-94e0-434a0054ce40 None None] MFA Rules not processed for user `05574eace38f4b6d945c7b1927fdadd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:32.762143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-da0d773d-0dab-4068-94e0-434a0054ce40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:32.766778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ba5e94d5f141b793bc5f59793b9274', 'request_id': 'req-0feb8499-ce08-4a15-9710-2df13d772cb6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:32.794307 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ba5e94d5f141b793bc5f59793b9274', 'request_id': 'req-0feb8499-ce08-4a15-9710-2df13d772cb6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:32.923059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] The token cache is being invalidate because project 91ba5e94d5f141b793bc5f59793b9274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:32.924114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:32.925342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0feb8499-ce08-4a15-9710-2df13d772cb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:32.931781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 853/1728] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:32 2022] DELETE /identity/v3/projects/91ba5e94d5f141b793bc5f59793b9274 => generated 0 bytes in 818 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:32.943031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a1c47-e924-4543-af14-54f9cb017c48 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:32.943459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a1c47-e924-4543-af14-54f9cb017c48 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:32.944368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a1c47-e924-4543-af14-54f9cb017c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:33.010895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d11a1c47-e924-4543-af14-54f9cb017c48 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:33.020913 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 876/1729] 10.222.0.71 () {58 vars in 1015 bytes} [Thu Jul 28 06:28:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:33.021452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d11a1c47-e924-4543-af14-54f9cb017c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:33.081557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d773d-0dab-4068-94e0-434a0054ce40 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:33.083626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-da0d773d-0dab-4068-94e0-434a0054ce40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:33.115418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d773d-0dab-4068-94e0-434a0054ce40 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:33.145079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c478a1-041e-49e0-9dfb-f0f5613a6fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:33.145807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c478a1-041e-49e0-9dfb-f0f5613a6fbf service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:33.146100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c478a1-041e-49e0-9dfb-f0f5613a6fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:33.146380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c478a1-041e-49e0-9dfb-f0f5613a6fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:33.164035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:33.182830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c478a1-041e-49e0-9dfb-f0f5613a6fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:33.186033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c478a1-041e-49e0-9dfb-f0f5613a6fbf service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:33.221867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 877/1730] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:33.269692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 854/1731] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:33.279494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556fc3f-55fe-4233-a625-4bcc625aa43f None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:33.283473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556fc3f-55fe-4233-a625-4bcc625aa43f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:33.283473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556fc3f-55fe-4233-a625-4bcc625aa43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:33.324393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e556fc3f-55fe-4233-a625-4bcc625aa43f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:33.327335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e556fc3f-55fe-4233-a625-4bcc625aa43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:33.395932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 878/1732] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:33.397175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a1c47-e924-4543-af14-54f9cb017c48 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:33.412080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a1c47-e924-4543-af14-54f9cb017c48 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:33.421946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1d61a-d028-454d-b5d0-26cc44383d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:33.422567 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1d61a-d028-454d-b5d0-26cc44383d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:33.422907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1d61a-d028-454d-b5d0-26cc44383d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:33.423194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1d61a-d028-454d-b5d0-26cc44383d74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:33.425081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1d61a-d028-454d-b5d0-26cc44383d74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:33.425416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1d61a-d028-454d-b5d0-26cc44383d74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:33.427733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1d61a-d028-454d-b5d0-26cc44383d74 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:33.449710 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 855/1733] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:33.450929 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556fc3f-55fe-4233-a625-4bcc625aa43f None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:33.485695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556fc3f-55fe-4233-a625-4bcc625aa43f None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:33.508876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51357e41-aa97-4d72-8e38-2d90e40ac9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:33.509907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51357e41-aa97-4d72-8e38-2d90e40ac9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:33.509907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51357e41-aa97-4d72-8e38-2d90e40ac9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:33.510125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51357e41-aa97-4d72-8e38-2d90e40ac9a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:33.510795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51357e41-aa97-4d72-8e38-2d90e40ac9a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:33.512797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51357e41-aa97-4d72-8e38-2d90e40ac9a4 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:33.669973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 879/1734] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:33 2022] POST /identity/v3/projects => generated 383 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:33.683783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1d61a-d028-454d-b5d0-26cc44383d74 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:33.698277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1d61a-d028-454d-b5d0-26cc44383d74 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:33.889322 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55be62-e16a-4759-885c-3dfb9a888bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:33.890318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55be62-e16a-4759-885c-3dfb9a888bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:33.890612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55be62-e16a-4759-885c-3dfb9a888bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:33.891024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55be62-e16a-4759-885c-3dfb9a888bb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:33.893383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55be62-e16a-4759-885c-3dfb9a888bb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:33.898924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55be62-e16a-4759-885c-3dfb9a888bb2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:34.056308 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 856/1735] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:33 2022] POST /identity/v3/users => generated 292 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:34.070299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51357e41-aa97-4d72-8e38-2d90e40ac9a4 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:34.089621 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51357e41-aa97-4d72-8e38-2d90e40ac9a4 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:34.109566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c2d90-a435-4810-97f7-b04dc5a82d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:34.112757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c2d90-a435-4810-97f7-b04dc5a82d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:34.115574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c2d90-a435-4810-97f7-b04dc5a82d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:34.116097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c2d90-a435-4810-97f7-b04dc5a82d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:34.117072 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c2d90-a435-4810-97f7-b04dc5a82d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:34.119059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c2d90-a435-4810-97f7-b04dc5a82d3c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:34.160648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55be62-e16a-4759-885c-3dfb9a888bb2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:34.181163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 880/1736] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:34.198241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55be62-e16a-4759-885c-3dfb9a888bb2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:34.215504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008c7bf-e720-4ee5-aa86-e3a0c193a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:34.216028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c7bf-e720-4ee5-aa86-e3a0c193a6da admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:34.216446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c7bf-e720-4ee5-aa86-e3a0c193a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:34.216813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c7bf-e720-4ee5-aa86-e3a0c193a6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:34.217624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008c7bf-e720-4ee5-aa86-e3a0c193a6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:34.220343 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008c7bf-e720-4ee5-aa86-e3a0c193a6da admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:34.280493 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 857/1737] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:28:34.284945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c2d90-a435-4810-97f7-b04dc5a82d3c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:34.297206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c2d90-a435-4810-97f7-b04dc5a82d3c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:34.318987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac990be-f3e1-4c6c-8cc9-ba5f9faf4e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:34.318987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac990be-f3e1-4c6c-8cc9-ba5f9faf4e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:34.318987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac990be-f3e1-4c6c-8cc9-ba5f9faf4e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:34.319241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac990be-f3e1-4c6c-8cc9-ba5f9faf4e38 admin admin] PATH_INFO: `/v3/projects/4a80a513867541e8a9dc8807f6d997b6/users/33eb5811f3e8477a98b75e1b5719968f/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:34.383900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac990be-f3e1-4c6c-8cc9-ba5f9faf4e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a80a513867541e8a9dc8807f6d997b6, user_id=33eb5811f3e8477a98b75e1b5719968f, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:34.389077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac990be-f3e1-4c6c-8cc9-ba5f9faf4e38 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:34.612987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 881/1738] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:34 2022] PUT /identity/v3/projects/4a80a513867541e8a9dc8807f6d997b6/users/33eb5811f3e8477a98b75e1b5719968f/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:34.630273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd49461-14e8-4796-8c09-4ef12dc4f558 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:34.630623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd49461-14e8-4796-8c09-4ef12dc4f558 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:34.630816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd49461-14e8-4796-8c09-4ef12dc4f558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:34.754637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fdd49461-14e8-4796-8c09-4ef12dc4f558 None None] MFA Rules not processed for user `33eb5811f3e8477a98b75e1b5719968f`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:34.762960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fdd49461-14e8-4796-8c09-4ef12dc4f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:34.845034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:35.040394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 858/1739] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:35.045462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac990be-f3e1-4c6c-8cc9-ba5f9faf4e38 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:35.309248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac990be-f3e1-4c6c-8cc9-ba5f9faf4e38 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:35.331678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08061d7f-1ddf-4b3c-9daf-b05e76272a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:35.332340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08061d7f-1ddf-4b3c-9daf-b05e76272a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:35.332627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08061d7f-1ddf-4b3c-9daf-b05e76272a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:35.332908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08061d7f-1ddf-4b3c-9daf-b05e76272a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:35.333633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08061d7f-1ddf-4b3c-9daf-b05e76272a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:35.335575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08061d7f-1ddf-4b3c-9daf-b05e76272a64 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:35.525189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 882/1740] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:35 2022] POST /identity/v3/projects => generated 381 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:35.529522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd49461-14e8-4796-8c09-4ef12dc4f558 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:35.554873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd49461-14e8-4796-8c09-4ef12dc4f558 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:35.785220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebf5ea-7b2a-42a3-a49d-c33ed0b62e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:35.786616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebf5ea-7b2a-42a3-a49d-c33ed0b62e37 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:35.786914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebf5ea-7b2a-42a3-a49d-c33ed0b62e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:35.787199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebf5ea-7b2a-42a3-a49d-c33ed0b62e37 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:35.790708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebf5ea-7b2a-42a3-a49d-c33ed0b62e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:35.800272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebf5ea-7b2a-42a3-a49d-c33ed0b62e37 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:35.955323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 859/1741] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:35 2022] POST /identity/v3/users => generated 297 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:35.960807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08061d7f-1ddf-4b3c-9daf-b05e76272a64 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:35.987668 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08061d7f-1ddf-4b3c-9daf-b05e76272a64 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:36.002480 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76a346-be68-43f4-9d23-b9e5828303fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:36.006206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76a346-be68-43f4-9d23-b9e5828303fa admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:36.006634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76a346-be68-43f4-9d23-b9e5828303fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:36.007707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76a346-be68-43f4-9d23-b9e5828303fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:36.009235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76a346-be68-43f4-9d23-b9e5828303fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:36.012325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76a346-be68-43f4-9d23-b9e5828303fa admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:36.052621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebf5ea-7b2a-42a3-a49d-c33ed0b62e37 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:36.061544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 883/1742] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:36.072083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebf5ea-7b2a-42a3-a49d-c33ed0b62e37 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:36.097014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da62a83-ac63-4215-a186-d03cebf5fb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:36.098078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da62a83-ac63-4215-a186-d03cebf5fb5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:36.099776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da62a83-ac63-4215-a186-d03cebf5fb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:36.100149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da62a83-ac63-4215-a186-d03cebf5fb5f admin admin] PATH_INFO: `/v3/projects/714ecf3dc0c747bdaeb9cdf6c2bdbd3c/users/808ca6803a25481da2b5397e2fee72fb/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:36.161181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da62a83-ac63-4215-a186-d03cebf5fb5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=714ecf3dc0c747bdaeb9cdf6c2bdbd3c, user_id=808ca6803a25481da2b5397e2fee72fb, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:36.165485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da62a83-ac63-4215-a186-d03cebf5fb5f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:36.279037 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 860/1743] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:36 2022] PUT /identity/v3/projects/714ecf3dc0c747bdaeb9cdf6c2bdbd3c/users/808ca6803a25481da2b5397e2fee72fb/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:28:36.284465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76a346-be68-43f4-9d23-b9e5828303fa admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:36.308653 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76a346-be68-43f4-9d23-b9e5828303fa admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:36.524207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f6b9f-047b-43a4-a702-a4dc2664646c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:36.525200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f6b9f-047b-43a4-a702-a4dc2664646c admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:36.525583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f6b9f-047b-43a4-a702-a4dc2664646c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:36.525868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f6b9f-047b-43a4-a702-a4dc2664646c admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:36.526590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f6b9f-047b-43a4-a702-a4dc2664646c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:36.529523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f6b9f-047b-43a4-a702-a4dc2664646c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:36.544838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:36.566676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 884/1744] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:36.571089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da62a83-ac63-4215-a186-d03cebf5fb5f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:36.589019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da62a83-ac63-4215-a186-d03cebf5fb5f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:36.605960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e13fdf-06ef-424d-b23d-beea2bf756b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:36.614676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e13fdf-06ef-424d-b23d-beea2bf756b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:36.614676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e13fdf-06ef-424d-b23d-beea2bf756b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:36.614676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e13fdf-06ef-424d-b23d-beea2bf756b6 admin admin] PATH_INFO: `/v3/projects/714ecf3dc0c747bdaeb9cdf6c2bdbd3c/users/808ca6803a25481da2b5397e2fee72fb/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:36.617660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e13fdf-06ef-424d-b23d-beea2bf756b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714ecf3dc0c747bdaeb9cdf6c2bdbd3c, user_id=808ca6803a25481da2b5397e2fee72fb, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:36.625869 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e13fdf-06ef-424d-b23d-beea2bf756b6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:36.687427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 861/1745] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:36 2022] PUT /identity/v3/projects/714ecf3dc0c747bdaeb9cdf6c2bdbd3c/users/808ca6803a25481da2b5397e2fee72fb/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:36.702897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a30f4e-a590-4722-96c1-8eb1d6c091bf None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:36.703867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a30f4e-a590-4722-96c1-8eb1d6c091bf None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:36.706014 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a30f4e-a590-4722-96c1-8eb1d6c091bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:36.832049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e9a30f4e-a590-4722-96c1-8eb1d6c091bf None None] MFA Rules not processed for user `808ca6803a25481da2b5397e2fee72fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:36.834911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e9a30f4e-a590-4722-96c1-8eb1d6c091bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:37.136879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 885/1746] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:37.143060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e13fdf-06ef-424d-b23d-beea2bf756b6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:37.146476 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b7e13fdf-06ef-424d-b23d-beea2bf756b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:37.331535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e13fdf-06ef-424d-b23d-beea2bf756b6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:37.344389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6b43e-7b2d-4f05-a0e4-927b3d09b704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:37.345129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6b43e-7b2d-4f05-a0e4-927b3d09b704 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:37.345994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6b43e-7b2d-4f05-a0e4-927b3d09b704 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:37.346285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6b43e-7b2d-4f05-a0e4-927b3d09b704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:37.532067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6b43e-7b2d-4f05-a0e4-927b3d09b704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:37.532067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6b43e-7b2d-4f05-a0e4-927b3d09b704 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:37.534747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a30f4e-a590-4722-96c1-8eb1d6c091bf None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:37.534747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e9a30f4e-a590-4722-96c1-8eb1d6c091bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:37.535879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 862/1747] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:37.622782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a30f4e-a590-4722-96c1-8eb1d6c091bf None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:37.636905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:37.637712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:37.637957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:37.638273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] PATH_INFO: `/v3/users/1144f2ad9639415f90f2dd71a6d91158` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:37.639908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1144f2ad9639415f90f2dd71a6d91158)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:37.643127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:37.703922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6b43e-7b2d-4f05-a0e4-927b3d09b704 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:37.706430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-efd6b43e-7b2d-4f05-a0e4-927b3d09b704 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:37.720752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6b43e-7b2d-4f05-a0e4-927b3d09b704 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:37.732430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f58bc1-a59d-42f1-aa8a-df611f1aba80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:37.733450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f58bc1-a59d-42f1-aa8a-df611f1aba80 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:37.733816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f58bc1-a59d-42f1-aa8a-df611f1aba80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:37.734007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f58bc1-a59d-42f1-aa8a-df611f1aba80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:37.763328 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:37.910686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1144f2ad9639415f90f2dd71a6d91158'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:37.940046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f58bc1-a59d-42f1-aa8a-df611f1aba80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:37.943918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f58bc1-a59d-42f1-aa8a-df611f1aba80 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:37.965860 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 863/1748] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:28:37.978053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1144f2ad9639415f90f2dd71a6d91158'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:37.979229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1144f2ad9639415f90f2dd71a6d91158'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:38.003429 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1144f2ad9639415f90f2dd71a6d91158'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:38.096855 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1144f2ad9639415f90f2dd71a6d91158'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:38.106911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 886/1749] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:37 2022] DELETE /identity/v3/users/1144f2ad9639415f90f2dd71a6d91158 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:38.113525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d73ea-0f73-441d-880c-fa0291dfe769 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:38.114159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d73ea-0f73-441d-880c-fa0291dfe769 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:38.114693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d73ea-0f73-441d-880c-fa0291dfe769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:38.194485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f98d73ea-0f73-441d-880c-fa0291dfe769 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:38.200663 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f98d73ea-0f73-441d-880c-fa0291dfe769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:38.467656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 864/1750] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:38.624412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:38.626254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:38.669199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:38.885761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d73ea-0f73-441d-880c-fa0291dfe769 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:38.888615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f98d73ea-0f73-441d-880c-fa0291dfe769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:38.916799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d73ea-0f73-441d-880c-fa0291dfe769 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:38.951001 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:38.952228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:38.952665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:38.953090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] PATH_INFO: `/v3/users/33eb5811f3e8477a98b75e1b5719968f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:38.954915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] RBAC: Authorizing `identity:delete_user(user_id=33eb5811f3e8477a98b75e1b5719968f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:38.957287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:39.009926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f860e93-37cc-411e-bf1a-2a2335e508f8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:39.043565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975617b9-fe28-455e-85e2-ec6c25b4f07e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:39.072908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975617b9-fe28-455e-85e2-ec6c25b4f07e service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:39.075042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975617b9-fe28-455e-85e2-ec6c25b4f07e service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:39.077194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975617b9-fe28-455e-85e2-ec6c25b4f07e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:39.159201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975617b9-fe28-455e-85e2-ec6c25b4f07e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:39.177900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975617b9-fe28-455e-85e2-ec6c25b4f07e service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:39.246208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 887/1751] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:39.249171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a657-72ee-492e-8292-eeba4b62e8e6 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:39.249519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a657-72ee-492e-8292-eeba4b62e8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:39.249955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a657-72ee-492e-8292-eeba4b62e8e6 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:39.254175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 888/1752] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:28:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:39.299760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33eb5811f3e8477a98b75e1b5719968f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:39.390102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33eb5811f3e8477a98b75e1b5719968f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:39.413399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33eb5811f3e8477a98b75e1b5719968f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:39.536880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33eb5811f3e8477a98b75e1b5719968f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:39.537782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-eada555a-d86d-4b54-84c3-ab462e44a91c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33eb5811f3e8477a98b75e1b5719968f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:39.551950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 865/1753] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:38 2022] DELETE /identity/v3/users/33eb5811f3e8477a98b75e1b5719968f => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:39.552046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d706d-bd60-41b2-8d76-4e1766e387c7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:39.552310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d706d-bd60-41b2-8d76-4e1766e387c7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:39.552834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d706d-bd60-41b2-8d76-4e1766e387c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:39.622910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898068d-097d-42d9-8c06-7da7bff682f1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:39.623345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898068d-097d-42d9-8c06-7da7bff682f1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:39.624294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898068d-097d-42d9-8c06-7da7bff682f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:39.651318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-a51d706d-bd60-41b2-8d76-4e1766e387c7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:39.654627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a51d706d-bd60-41b2-8d76-4e1766e387c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:39.708471 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6898068d-097d-42d9-8c06-7da7bff682f1 None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:39.717176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6898068d-097d-42d9-8c06-7da7bff682f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:39.781178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:39.906958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 889/1754] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:39.922332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53b17b-3dda-49ee-ba39-070ef9a8edb6 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:39.923016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53b17b-3dda-49ee-ba39-070ef9a8edb6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:39.924563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53b17b-3dda-49ee-ba39-070ef9a8edb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:39.932918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 866/1755] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:28:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:39.934961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898068d-097d-42d9-8c06-7da7bff682f1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:39.937245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6898068d-097d-42d9-8c06-7da7bff682f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:39.984207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-5e53b17b-3dda-49ee-ba39-070ef9a8edb6 None None] MFA Rules not processed for user `61ad8186d91848bc8be42e3be51e9a75`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:39.984207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-5e53b17b-3dda-49ee-ba39-070ef9a8edb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:40.073778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898068d-097d-42d9-8c06-7da7bff682f1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:40.080569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f889b-ec80-479c-b7e7-c5bef969c3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:40.081443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f889b-ec80-479c-b7e7-c5bef969c3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:40.081847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f889b-ec80-479c-b7e7-c5bef969c3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:40.082301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f889b-ec80-479c-b7e7-c5bef969c3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:40.115222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f889b-ec80-479c-b7e7-c5bef969c3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:40.117048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f889b-ec80-479c-b7e7-c5bef969c3c2 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:40.144344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 867/1756] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:40.159282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f889b-ec80-479c-b7e7-c5bef969c3c2 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:40.162114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b03f889b-ec80-479c-b7e7-c5bef969c3c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:40.175595 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 890/1757] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:40.206078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f889b-ec80-479c-b7e7-c5bef969c3c2 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:40.223374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caa75a-d0cd-4b50-b987-3845a284a1f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:40.224220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caa75a-d0cd-4b50-b987-3845a284a1f7 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:40.224507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caa75a-d0cd-4b50-b987-3845a284a1f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:40.224788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caa75a-d0cd-4b50-b987-3845a284a1f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:40.253768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caa75a-d0cd-4b50-b987-3845a284a1f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:40.260848 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caa75a-d0cd-4b50-b987-3845a284a1f7 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:40.289933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 868/1758] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:41.815647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53b17b-3dda-49ee-ba39-070ef9a8edb6 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:41.818071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-5e53b17b-3dda-49ee-ba39-070ef9a8edb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:41.872022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53b17b-3dda-49ee-ba39-070ef9a8edb6 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:41.892829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:41.892829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:41.892829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:41.892829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] PATH_INFO: `/v3/projects/4a80a513867541e8a9dc8807f6d997b6` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:41.902292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a80a513867541e8a9dc8807f6d997b6)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:41.904715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:42.089432 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92f7e4-cca1-4093-aea6-04c89cfc4717 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:42.091174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92f7e4-cca1-4093-aea6-04c89cfc4717 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:42.091678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92f7e4-cca1-4093-aea6-04c89cfc4717 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:42.094099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 869/1759] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:28:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:42.103536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:42.105859 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a80a513867541e8a9dc8807f6d997b6', 'request_id': 'req-da267133-6bb3-4df0-ab69-f1c7b2311716'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:42.121121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a80a513867541e8a9dc8807f6d997b6', 'request_id': 'req-da267133-6bb3-4df0-ab69-f1c7b2311716'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:42.169511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3045e-8895-4c6e-aab1-aa4f5383a83f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:42.170105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3045e-8895-4c6e-aab1-aa4f5383a83f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:42.170612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3045e-8895-4c6e-aab1-aa4f5383a83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:42.211831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a80a513867541e8a9dc8807f6d997b6', 'request_id': 'req-da267133-6bb3-4df0-ab69-f1c7b2311716'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:42.235325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] The token cache is being invalidate because project 4a80a513867541e8a9dc8807f6d997b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:42.235638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:42.237530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:42.242349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 891/1760] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:41 2022] DELETE /identity/v3/projects/4a80a513867541e8a9dc8807f6d997b6 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:42.244317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b0f3045e-8895-4c6e-aab1-aa4f5383a83f None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:42.245816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b0f3045e-8895-4c6e-aab1-aa4f5383a83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:42.247608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:42.249131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:42.451339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 870/1761] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:28:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 28 06:28:42.478337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da267133-6bb3-4df0-ab69-f1c7b2311716 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:42.484410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:42.485195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:42.485664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:42.486058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] PATH_INFO: `/v3/users/808ca6803a25481da2b5397e2fee72fb` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:42.487739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=808ca6803a25481da2b5397e2fee72fb)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:42.490097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:42.726203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '808ca6803a25481da2b5397e2fee72fb'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:42.751612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '808ca6803a25481da2b5397e2fee72fb'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:42.752752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '808ca6803a25481da2b5397e2fee72fb'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:42.762435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '808ca6803a25481da2b5397e2fee72fb'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:42.840135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '808ca6803a25481da2b5397e2fee72fb'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:42.849804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 892/1762] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:42 2022] DELETE /identity/v3/users/808ca6803a25481da2b5397e2fee72fb => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:43.578123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3045e-8895-4c6e-aab1-aa4f5383a83f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:43.581821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b0f3045e-8895-4c6e-aab1-aa4f5383a83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:43.601089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3045e-8895-4c6e-aab1-aa4f5383a83f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:43.654806 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:43.694079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:43.695777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:43.696420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:43.696934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] PATH_INFO: `/v3/projects/6ec77a76d1da455b9a2483a60421e699` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:43.698684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ec77a76d1da455b9a2483a60421e699)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:43.701421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:43.858000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ec77a76d1da455b9a2483a60421e699', 'request_id': 'req-68beb8c4-2466-4603-8d0b-b7e93ee0d094'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:43.873768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ec77a76d1da455b9a2483a60421e699', 'request_id': 'req-68beb8c4-2466-4603-8d0b-b7e93ee0d094'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:43.888284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ec77a76d1da455b9a2483a60421e699', 'request_id': 'req-68beb8c4-2466-4603-8d0b-b7e93ee0d094'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.029051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] The token cache is being invalidate because project 6ec77a76d1da455b9a2483a60421e699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:44.029486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.031092 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.040880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 871/1763] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:43 2022] DELETE /identity/v3/projects/6ec77a76d1da455b9a2483a60421e699 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:44.044518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:44.049658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:44.084286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc1af4-8d6e-46b1-9f12-d603b118c69c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:44.089654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:44.091637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:44.165782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:44.179955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:44.179955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:44.179955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:44.179955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] PATH_INFO: `/v3/users/cf724782aa204022a5f35fae4a9aa684` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:44.185179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf724782aa204022a5f35fae4a9aa684)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:44.192957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:44.272002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68beb8c4-2466-4603-8d0b-b7e93ee0d094 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:44.294666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:44.294666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:44.294666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:44.294666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] PATH_INFO: `/v3/projects/714ecf3dc0c747bdaeb9cdf6c2bdbd3c` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:44.295123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=714ecf3dc0c747bdaeb9cdf6c2bdbd3c)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:44.295750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:44.461419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf724782aa204022a5f35fae4a9aa684'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.479086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714ecf3dc0c747bdaeb9cdf6c2bdbd3c', 'request_id': 'req-c5266e10-e66c-4582-bc46-ff6e467dc213'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.489469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714ecf3dc0c747bdaeb9cdf6c2bdbd3c', 'request_id': 'req-c5266e10-e66c-4582-bc46-ff6e467dc213'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.496810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714ecf3dc0c747bdaeb9cdf6c2bdbd3c', 'request_id': 'req-c5266e10-e66c-4582-bc46-ff6e467dc213'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.501381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:44.505565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf724782aa204022a5f35fae4a9aa684'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.506902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf724782aa204022a5f35fae4a9aa684'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.528072 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf724782aa204022a5f35fae4a9aa684'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.579281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] The token cache is being invalidate because project 714ecf3dc0c747bdaeb9cdf6c2bdbd3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:44.579871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.580467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.586789 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 872/1764] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:44 2022] DELETE /identity/v3/projects/714ecf3dc0c747bdaeb9cdf6c2bdbd3c => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:44.587396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:44.588882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:44.675064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-72028dac-f0ae-4e2f-94f5-3b346040123b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf724782aa204022a5f35fae4a9aa684'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:44.684552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 893/1765] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:44 2022] DELETE /identity/v3/users/cf724782aa204022a5f35fae4a9aa684 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:44.702930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6927-eb4e-48a2-b3b3-4029ba71cb10 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:44.706675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6927-eb4e-48a2-b3b3-4029ba71cb10 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:44.706675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6927-eb4e-48a2-b3b3-4029ba71cb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:44.768786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5266e10-e66c-4582-bc46-ff6e467dc213 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:44.784506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-aebf6927-eb4e-48a2-b3b3-4029ba71cb10 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:44.788949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-aebf6927-eb4e-48a2-b3b3-4029ba71cb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:44.942479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 894/1766] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:44.944904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc8f5f-320b-4d3e-99a0-f2ec86dfb866 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:44.945488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc8f5f-320b-4d3e-99a0-f2ec86dfb866 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:44.945945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc8f5f-320b-4d3e-99a0-f2ec86dfb866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:44.973091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-afdc8f5f-320b-4d3e-99a0-f2ec86dfb866 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:44.979714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-afdc8f5f-320b-4d3e-99a0-f2ec86dfb866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:44.980524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662095e0-8ee8-44b9-9554-951207598ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:44.980524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662095e0-8ee8-44b9-9554-951207598ded service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:44.980524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662095e0-8ee8-44b9-9554-951207598ded service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:44.980524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662095e0-8ee8-44b9-9554-951207598ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:44.997661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 895/1767] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:44.998530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc8f5f-320b-4d3e-99a0-f2ec86dfb866 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:45.000365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-662095e0-8ee8-44b9-9554-951207598ded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:45.023832 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc8f5f-320b-4d3e-99a0-f2ec86dfb866 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:45.024965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2ce81-a6c2-41a9-8c75-53760e89b25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:45.027037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2ce81-a6c2-41a9-8c75-53760e89b25b admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:45.027037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2ce81-a6c2-41a9-8c75-53760e89b25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:45.027224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2ce81-a6c2-41a9-8c75-53760e89b25b admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:45.028581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2ce81-a6c2-41a9-8c75-53760e89b25b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:45.028823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2ce81-a6c2-41a9-8c75-53760e89b25b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:45.030578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2ce81-a6c2-41a9-8c75-53760e89b25b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:45.035814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662095e0-8ee8-44b9-9554-951207598ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:45.038290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662095e0-8ee8-44b9-9554-951207598ded service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:45.069149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 896/1768] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:45.076285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2ce81-a6c2-41a9-8c75-53760e89b25b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:45.081442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 873/1769] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:45.118883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2ce81-a6c2-41a9-8c75-53760e89b25b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:45.128285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb8aba-dc1d-4968-a491-0da6e0937ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:45.129213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb8aba-dc1d-4968-a491-0da6e0937ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:45.129786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb8aba-dc1d-4968-a491-0da6e0937ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:45.130213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb8aba-dc1d-4968-a491-0da6e0937ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:45.131511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb8aba-dc1d-4968-a491-0da6e0937ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:45.136819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb8aba-dc1d-4968-a491-0da6e0937ea6 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:45.312285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 897/1770] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:45 2022] POST /identity/v3/projects => generated 375 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:45.316557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662095e0-8ee8-44b9-9554-951207598ded service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:45.335859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662095e0-8ee8-44b9-9554-951207598ded service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:45.487770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ed288-2548-4575-9d44-ed53267ef8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:45.487770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ed288-2548-4575-9d44-ed53267ef8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:45.487770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ed288-2548-4575-9d44-ed53267ef8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:45.487770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ed288-2548-4575-9d44-ed53267ef8e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:45.488188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ed288-2548-4575-9d44-ed53267ef8e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:45.488188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ed288-2548-4575-9d44-ed53267ef8e7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:45.507366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb8aba-dc1d-4968-a491-0da6e0937ea6 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:45.513886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-82fb8aba-dc1d-4968-a491-0da6e0937ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:45.540107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb8aba-dc1d-4968-a491-0da6e0937ea6 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:45.558071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:45.558907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:45.559196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:45.559479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] PATH_INFO: `/v3/projects/8e03f432f7bf43a3a418138e69e96f0c` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:45.566603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e03f432f7bf43a3a418138e69e96f0c)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:45.568557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:45.634523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 874/1771] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:45 2022] POST /identity/v3/users => generated 288 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:45.635504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ed288-2548-4575-9d44-ed53267ef8e7 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:45.652458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ed288-2548-4575-9d44-ed53267ef8e7 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:45.662411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30527f60-5f80-4256-be38-bb20c8cfeb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:45.663060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30527f60-5f80-4256-be38-bb20c8cfeb02 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:45.663407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30527f60-5f80-4256-be38-bb20c8cfeb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:45.663689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30527f60-5f80-4256-be38-bb20c8cfeb02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:45.664401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30527f60-5f80-4256-be38-bb20c8cfeb02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:45.666123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30527f60-5f80-4256-be38-bb20c8cfeb02 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:45.685017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 875/1772] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:45.690267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45496a-74fc-44f1-86fc-1383ab97f955 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:45.690267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45496a-74fc-44f1-86fc-1383ab97f955 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:45.690267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45496a-74fc-44f1-86fc-1383ab97f955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:45.709668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7b45496a-74fc-44f1-86fc-1383ab97f955 None None] MFA Rules not processed for user `6d4528589ffa478689e905d47e4c37ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:45.711564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7b45496a-74fc-44f1-86fc-1383ab97f955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:45.734523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e03f432f7bf43a3a418138e69e96f0c', 'request_id': 'req-08752c4f-3e9c-4b89-85e5-4354537aebc7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:45.740931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e03f432f7bf43a3a418138e69e96f0c', 'request_id': 'req-08752c4f-3e9c-4b89-85e5-4354537aebc7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:45.811349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 876/1773] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:45.811349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45496a-74fc-44f1-86fc-1383ab97f955 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:45.828789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e03f432f7bf43a3a418138e69e96f0c', 'request_id': 'req-08752c4f-3e9c-4b89-85e5-4354537aebc7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:45.845239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] The token cache is being invalidate because project 8e03f432f7bf43a3a418138e69e96f0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:45.845660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:45.846373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-08752c4f-3e9c-4b89-85e5-4354537aebc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:45.848348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 898/1774] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:45 2022] DELETE /identity/v3/projects/8e03f432f7bf43a3a418138e69e96f0c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:28:45.854532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb297b8-2cb9-4eeb-9947-f5d67dade524 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:45.855935 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb297b8-2cb9-4eeb-9947-f5d67dade524 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:45.856990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb297b8-2cb9-4eeb-9947-f5d67dade524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:45.902537 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-8fb297b8-2cb9-4eeb-9947-f5d67dade524 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:45.905169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8fb297b8-2cb9-4eeb-9947-f5d67dade524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:45.933796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45496a-74fc-44f1-86fc-1383ab97f955 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:45.941575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281fe0f-ef18-427b-ae14-d0c3d33fe15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:45.944683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281fe0f-ef18-427b-ae14-d0c3d33fe15e admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:45.944982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281fe0f-ef18-427b-ae14-d0c3d33fe15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:45.945267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281fe0f-ef18-427b-ae14-d0c3d33fe15e admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:45.946130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281fe0f-ef18-427b-ae14-d0c3d33fe15e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:45.947801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281fe0f-ef18-427b-ae14-d0c3d33fe15e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:45.967292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 877/1775] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:45.979231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281fe0f-ef18-427b-ae14-d0c3d33fe15e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:45.979754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5281fe0f-ef18-427b-ae14-d0c3d33fe15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:45.999897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281fe0f-ef18-427b-ae14-d0c3d33fe15e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:46.007401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abf2e8-f761-4650-8b4c-a733365587e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:46.009908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abf2e8-f761-4650-8b4c-a733365587e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.009908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abf2e8-f761-4650-8b4c-a733365587e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.009908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abf2e8-f761-4650-8b4c-a733365587e4 admin admin] PATH_INFO: `/v3/projects/77ff6eefb6164ff8a16a0988f9667420/users/ce89c4a504bb4601b6ec8c6ad32c2aa0/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.025345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 899/1776] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:46.028759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:46.039293 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abf2e8-f761-4650-8b4c-a733365587e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ff6eefb6164ff8a16a0988f9667420, user_id=ce89c4a504bb4601b6ec8c6ad32c2aa0, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:46.041348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6eb8b-c6b0-4b08-a711-4442eae08772 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.042026 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6eb8b-c6b0-4b08-a711-4442eae08772 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.042555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6eb8b-c6b0-4b08-a711-4442eae08772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.043285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abf2e8-f761-4650-8b4c-a733365587e4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:46.077591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-6db6eb8b-c6b0-4b08-a711-4442eae08772 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:46.080556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6db6eb8b-c6b0-4b08-a711-4442eae08772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:46.103226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 900/1777] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:46.104379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6eb8b-c6b0-4b08-a711-4442eae08772 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:46.117203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6eb8b-c6b0-4b08-a711-4442eae08772 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:46.124667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a7a7b-a780-4479-8f29-b75cce65c997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:46.125600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a7a7b-a780-4479-8f29-b75cce65c997 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.127106 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a7a7b-a780-4479-8f29-b75cce65c997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.127732 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a7a7b-a780-4479-8f29-b75cce65c997 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.129759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a7a7b-a780-4479-8f29-b75cce65c997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:46.130315 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a7a7b-a780-4479-8f29-b75cce65c997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:46.133706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a7a7b-a780-4479-8f29-b75cce65c997 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:46.157792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 878/1778] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:45 2022] PUT /identity/v3/projects/77ff6eefb6164ff8a16a0988f9667420/users/ce89c4a504bb4601b6ec8c6ad32c2aa0/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:46.159603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 901/1779] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:46.166983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74be9e0-6d11-4df2-82e1-d208b845b0a5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.169692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74be9e0-6d11-4df2-82e1-d208b845b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.170621 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a7a7b-a780-4479-8f29-b75cce65c997 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:46.171296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74be9e0-6d11-4df2-82e1-d208b845b0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.190236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a7a7b-a780-4479-8f29-b75cce65c997 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:46.251691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f74be9e0-6d11-4df2-82e1-d208b845b0a5 None None] MFA Rules not processed for user `ce89c4a504bb4601b6ec8c6ad32c2aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:46.253479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f74be9e0-6d11-4df2-82e1-d208b845b0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:46.292901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4355ec7-851c-4033-ba93-1284150c216f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:46.295091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4355ec7-851c-4033-ba93-1284150c216f admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.295561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4355ec7-851c-4033-ba93-1284150c216f admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.296558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4355ec7-851c-4033-ba93-1284150c216f admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.301864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4355ec7-851c-4033-ba93-1284150c216f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:46.310577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4355ec7-851c-4033-ba93-1284150c216f admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:46.383552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 879/1780] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:46.384564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74be9e0-6d11-4df2-82e1-d208b845b0a5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:46.403347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74be9e0-6d11-4df2-82e1-d208b845b0a5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:46.409397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10630a-848e-404d-ad9c-47fe02871fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:46.411987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10630a-848e-404d-ad9c-47fe02871fde admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.412921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10630a-848e-404d-ad9c-47fe02871fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.413738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10630a-848e-404d-ad9c-47fe02871fde admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.414836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10630a-848e-404d-ad9c-47fe02871fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:46.417703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10630a-848e-404d-ad9c-47fe02871fde admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:46.434858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 902/1781] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:46 2022] POST /identity/v3/projects => generated 387 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:46.440423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4355ec7-851c-4033-ba93-1284150c216f admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:46.543447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4355ec7-851c-4033-ba93-1284150c216f admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:46.551632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3f15c-c4b6-48d7-9fdd-1df0c00f8376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:46.552391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3f15c-c4b6-48d7-9fdd-1df0c00f8376 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.553186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3f15c-c4b6-48d7-9fdd-1df0c00f8376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.553717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3f15c-c4b6-48d7-9fdd-1df0c00f8376 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.556156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3f15c-c4b6-48d7-9fdd-1df0c00f8376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:46.559345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3f15c-c4b6-48d7-9fdd-1df0c00f8376 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:46.572958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:46.593526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 880/1782] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:46 2022] POST /identity/v3/projects => generated 375 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:46.598458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10630a-848e-404d-ad9c-47fe02871fde admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:46.647203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 903/1783] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:46 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:28:46.652819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3f15c-c4b6-48d7-9fdd-1df0c00f8376 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:46.678461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3f15c-c4b6-48d7-9fdd-1df0c00f8376 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:46.689584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5968e71-ae75-4659-a565-d02baff2b04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:46.690344 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5968e71-ae75-4659-a565-d02baff2b04a admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.690739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5968e71-ae75-4659-a565-d02baff2b04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.691124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5968e71-ae75-4659-a565-d02baff2b04a admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.691962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5968e71-ae75-4659-a565-d02baff2b04a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:46.694697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10630a-848e-404d-ad9c-47fe02871fde admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:46.695189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5968e71-ae75-4659-a565-d02baff2b04a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:46.701011 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f84e9a-4ef1-4f67-9c97-52def55e4bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:46.703362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f84e9a-4ef1-4f67-9c97-52def55e4bff admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.703868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f84e9a-4ef1-4f67-9c97-52def55e4bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.705130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f84e9a-4ef1-4f67-9c97-52def55e4bff admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.706625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f84e9a-4ef1-4f67-9c97-52def55e4bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:46.708966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f84e9a-4ef1-4f67-9c97-52def55e4bff admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:46.709100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 904/1784] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:46.715035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5968e71-ae75-4659-a565-d02baff2b04a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:46.731527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5968e71-ae75-4659-a565-d02baff2b04a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:46.737963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04e5c9-bde5-44bd-93a2-b3d1bd6b7eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:46.738715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04e5c9-bde5-44bd-93a2-b3d1bd6b7eee admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.739587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04e5c9-bde5-44bd-93a2-b3d1bd6b7eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.739918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04e5c9-bde5-44bd-93a2-b3d1bd6b7eee admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.740928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04e5c9-bde5-44bd-93a2-b3d1bd6b7eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:46.742877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04e5c9-bde5-44bd-93a2-b3d1bd6b7eee admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:46.758744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 905/1785] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:46.767418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04e5c9-bde5-44bd-93a2-b3d1bd6b7eee admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:46.788975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04e5c9-bde5-44bd-93a2-b3d1bd6b7eee admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:46.794441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12865e9c-24a3-4b50-9568-fe9596714952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:46.849971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12865e9c-24a3-4b50-9568-fe9596714952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.849971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12865e9c-24a3-4b50-9568-fe9596714952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.849971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12865e9c-24a3-4b50-9568-fe9596714952 admin admin] PATH_INFO: `/v3/projects/81a3fce7cb1447e3979d9a2313f54dd2/users/9c065f5c6b61406f99cb3b0aa86d0974/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.855781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12865e9c-24a3-4b50-9568-fe9596714952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a3fce7cb1447e3979d9a2313f54dd2, user_id=9c065f5c6b61406f99cb3b0aa86d0974, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:46.861599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12865e9c-24a3-4b50-9568-fe9596714952 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:47.031717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 881/1786] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:46 2022] POST /identity/v3/users => generated 288 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:47.035893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f84e9a-4ef1-4f67-9c97-52def55e4bff admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:47.049429 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f84e9a-4ef1-4f67-9c97-52def55e4bff admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:47.055377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c45657-825e-4764-b9df-58a056227fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:47.056034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c45657-825e-4764-b9df-58a056227fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.056323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c45657-825e-4764-b9df-58a056227fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.056607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c45657-825e-4764-b9df-58a056227fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.057725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c45657-825e-4764-b9df-58a056227fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:47.062046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c45657-825e-4764-b9df-58a056227fa5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:47.075615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 882/1787] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:28:47.078978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c45657-825e-4764-b9df-58a056227fa5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:47.088297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 906/1788] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:46 2022] PUT /identity/v3/projects/81a3fce7cb1447e3979d9a2313f54dd2/users/9c065f5c6b61406f99cb3b0aa86d0974/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:47.091620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c45657-825e-4764-b9df-58a056227fa5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:47.100192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783aab0a-daf5-4359-92ed-4913d5efd3da None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.100404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783aab0a-daf5-4359-92ed-4913d5efd3da None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.100701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783aab0a-daf5-4359-92ed-4913d5efd3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.144302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-783aab0a-daf5-4359-92ed-4913d5efd3da None None] MFA Rules not processed for user `9c065f5c6b61406f99cb3b0aa86d0974`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:47.145647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-783aab0a-daf5-4359-92ed-4913d5efd3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:47.205642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdcb23-3a2d-4e37-a2d3-16a9ec3362e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:47.206568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdcb23-3a2d-4e37-a2d3-16a9ec3362e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.206568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdcb23-3a2d-4e37-a2d3-16a9ec3362e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.206964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdcb23-3a2d-4e37-a2d3-16a9ec3362e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.207882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdcb23-3a2d-4e37-a2d3-16a9ec3362e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:47.210621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdcb23-3a2d-4e37-a2d3-16a9ec3362e4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:47.230678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 883/1789] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:47.230678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdcb23-3a2d-4e37-a2d3-16a9ec3362e4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:47.240490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdcb23-3a2d-4e37-a2d3-16a9ec3362e4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:47.240777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 907/1790] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:47.245388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783aab0a-daf5-4359-92ed-4913d5efd3da None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:47.248193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a84451-9066-461a-9990-7d96450a0f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:47.248901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a84451-9066-461a-9990-7d96450a0f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.249398 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a84451-9066-461a-9990-7d96450a0f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.254415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a84451-9066-461a-9990-7d96450a0f16 admin admin] PATH_INFO: `/v3/projects/8a3320422a6c458894aa127593f8eb29/users/7d88710b8e38417ab8385bc511dec253/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.259172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783aab0a-daf5-4359-92ed-4913d5efd3da None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:47.264721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f051044-a253-452d-b83d-e00fa149308e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:47.265532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f051044-a253-452d-b83d-e00fa149308e admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.265882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f051044-a253-452d-b83d-e00fa149308e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.266235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f051044-a253-452d-b83d-e00fa149308e admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.273493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f051044-a253-452d-b83d-e00fa149308e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:47.277437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f051044-a253-452d-b83d-e00fa149308e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:47.288084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a84451-9066-461a-9990-7d96450a0f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a3320422a6c458894aa127593f8eb29, user_id=7d88710b8e38417ab8385bc511dec253, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:47.295479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a84451-9066-461a-9990-7d96450a0f16 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:47.451490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 884/1791] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:47 2022] PUT /identity/v3/projects/8a3320422a6c458894aa127593f8eb29/users/7d88710b8e38417ab8385bc511dec253/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:28:47.456831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 908/1792] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:47 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:28:47.458817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a84451-9066-461a-9990-7d96450a0f16 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:47.461781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a46f9b-9e63-4ef4-8cb1-c4fe81632bd1 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.462154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a46f9b-9e63-4ef4-8cb1-c4fe81632bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.462521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a46f9b-9e63-4ef4-8cb1-c4fe81632bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.498214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-45a46f9b-9e63-4ef4-8cb1-c4fe81632bd1 None None] MFA Rules not processed for user `7d88710b8e38417ab8385bc511dec253`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:47.499469 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-45a46f9b-9e63-4ef4-8cb1-c4fe81632bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:47.560673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a84451-9066-461a-9990-7d96450a0f16 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:47.615419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 909/1793] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:47.617931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed1bf2-e442-49bf-9f72-8a03f3fb315b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.618419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed1bf2-e442-49bf-9f72-8a03f3fb315b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.618834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed1bf2-e442-49bf-9f72-8a03f3fb315b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.631032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84900668-1598-44a5-865f-361634aadd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:47.631765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84900668-1598-44a5-865f-361634aadd8d admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.632059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84900668-1598-44a5-865f-361634aadd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.632344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84900668-1598-44a5-865f-361634aadd8d admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.633025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84900668-1598-44a5-865f-361634aadd8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:47.635233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84900668-1598-44a5-865f-361634aadd8d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:47.644880 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-7fed1bf2-e442-49bf-9f72-8a03f3fb315b None None] MFA Rules not processed for user `0d268e37c8b64bcc9261a80bb0461edc`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:47.646145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7fed1bf2-e442-49bf-9f72-8a03f3fb315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:47.727286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 910/1794] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:47.733438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed1bf2-e442-49bf-9f72-8a03f3fb315b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:47.740512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed1bf2-e442-49bf-9f72-8a03f3fb315b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:47.758371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97622b-9666-476e-a430-e769fbceb59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:47.762144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97622b-9666-476e-a430-e769fbceb59e admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.762144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97622b-9666-476e-a430-e769fbceb59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.762144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97622b-9666-476e-a430-e769fbceb59e admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.762438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 885/1795] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:47 2022] POST /identity/v3/users => generated 300 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:47.762873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97622b-9666-476e-a430-e769fbceb59e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:47.762873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97622b-9666-476e-a430-e769fbceb59e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:47.764818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84900668-1598-44a5-865f-361634aadd8d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:47.776416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84900668-1598-44a5-865f-361634aadd8d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:47.782194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac5922-53d7-4d7a-844f-5de8a81a57b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:47.782861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac5922-53d7-4d7a-844f-5de8a81a57b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.783152 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac5922-53d7-4d7a-844f-5de8a81a57b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.783436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac5922-53d7-4d7a-844f-5de8a81a57b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.784250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac5922-53d7-4d7a-844f-5de8a81a57b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:47.785912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac5922-53d7-4d7a-844f-5de8a81a57b4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:47.800015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 886/1796] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:28:47.804916 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac5922-53d7-4d7a-844f-5de8a81a57b4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:47.818219 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac5922-53d7-4d7a-844f-5de8a81a57b4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:47.823538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd887c-badb-4751-aff7-43b0affd53a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:47.824201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd887c-badb-4751-aff7-43b0affd53a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:47.824491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd887c-badb-4751-aff7-43b0affd53a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:47.825485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd887c-badb-4751-aff7-43b0affd53a6 admin admin] PATH_INFO: `/v3/projects/e647475285ff45419de2a25cb0cb482c/users/825658bff3e747fca86db2b3862f020b/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:47.851874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:47.863561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd887c-badb-4751-aff7-43b0affd53a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e647475285ff45419de2a25cb0cb482c, user_id=825658bff3e747fca86db2b3862f020b, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:47.863561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd887c-badb-4751-aff7-43b0affd53a6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:47.864148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 911/1797] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:47 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:47.869438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97622b-9666-476e-a430-e769fbceb59e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:47.944540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 887/1798] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:47 2022] PUT /identity/v3/projects/e647475285ff45419de2a25cb0cb482c/users/825658bff3e747fca86db2b3862f020b/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:28:47.946776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd887c-badb-4751-aff7-43b0affd53a6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:47.964737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:47.989903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97622b-9666-476e-a430-e769fbceb59e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:48.087225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccbe74-2d83-4eae-bd13-9159f451bf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:48.088311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccbe74-2d83-4eae-bd13-9159f451bf4a admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.089247 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccbe74-2d83-4eae-bd13-9159f451bf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.089747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccbe74-2d83-4eae-bd13-9159f451bf4a admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.091022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccbe74-2d83-4eae-bd13-9159f451bf4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:48.095473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccbe74-2d83-4eae-bd13-9159f451bf4a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:48.109014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd887c-badb-4751-aff7-43b0affd53a6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:48.126229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139a109-ac8f-4551-97b8-18cfe1526560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:48.127032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139a109-ac8f-4551-97b8-18cfe1526560 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.127608 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139a109-ac8f-4551-97b8-18cfe1526560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.127733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139a109-ac8f-4551-97b8-18cfe1526560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.128513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139a109-ac8f-4551-97b8-18cfe1526560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:48.131197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139a109-ac8f-4551-97b8-18cfe1526560 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:48.147614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 888/1799] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:48.149421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139a109-ac8f-4551-97b8-18cfe1526560 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:48.161852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139a109-ac8f-4551-97b8-18cfe1526560 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:48.170272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac92438-2eef-43f2-8334-4e02758ad0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:48.171062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac92438-2eef-43f2-8334-4e02758ad0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.171428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac92438-2eef-43f2-8334-4e02758ad0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.171787 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac92438-2eef-43f2-8334-4e02758ad0ae admin admin] PATH_INFO: `/v3/projects/e647475285ff45419de2a25cb0cb482c/users/825658bff3e747fca86db2b3862f020b/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.174089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac92438-2eef-43f2-8334-4e02758ad0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e647475285ff45419de2a25cb0cb482c, user_id=825658bff3e747fca86db2b3862f020b, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:48.177170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac92438-2eef-43f2-8334-4e02758ad0ae admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:48.196777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 889/1800] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:48 2022] PUT /identity/v3/projects/e647475285ff45419de2a25cb0cb482c/users/825658bff3e747fca86db2b3862f020b/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:48.211621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f265758-b12c-441e-aac3-fac7966a797d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.211621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f265758-b12c-441e-aac3-fac7966a797d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.211621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f265758-b12c-441e-aac3-fac7966a797d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.221119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 912/1801] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:47 2022] POST /identity/v3/users => generated 295 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:48.222356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccbe74-2d83-4eae-bd13-9159f451bf4a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:48.240114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccbe74-2d83-4eae-bd13-9159f451bf4a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:48.251274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3f265758-b12c-441e-aac3-fac7966a797d None None] MFA Rules not processed for user `825658bff3e747fca86db2b3862f020b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:48.254347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3f265758-b12c-441e-aac3-fac7966a797d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:48.360862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce87de-5b8e-4d65-b857-d91053eb9206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:48.371971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce87de-5b8e-4d65-b857-d91053eb9206 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.372267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce87de-5b8e-4d65-b857-d91053eb9206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.372546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce87de-5b8e-4d65-b857-d91053eb9206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.373380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce87de-5b8e-4d65-b857-d91053eb9206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:48.377675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce87de-5b8e-4d65-b857-d91053eb9206 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:48.386576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 890/1802] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:48.395809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 913/1803] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:28:48.402045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f265758-b12c-441e-aac3-fac7966a797d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:48.410511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce87de-5b8e-4d65-b857-d91053eb9206 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:48.414813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d7ce87de-5b8e-4d65-b857-d91053eb9206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:48.426271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f265758-b12c-441e-aac3-fac7966a797d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:48.436399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:48.437998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d84117-8add-4685-a3c1-3891505e74b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:48.438985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d84117-8add-4685-a3c1-3891505e74b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.439396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d84117-8add-4685-a3c1-3891505e74b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.439683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d84117-8add-4685-a3c1-3891505e74b6 admin admin] PATH_INFO: `/v3/projects/0948b03ac360454490482c4a7ef7c7ae/users/1d1f0a3c098f48c2a17d01a7a10129e4/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.456790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d84117-8add-4685-a3c1-3891505e74b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0948b03ac360454490482c4a7ef7c7ae, user_id=1d1f0a3c098f48c2a17d01a7a10129e4, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:48.459510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d84117-8add-4685-a3c1-3891505e74b6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:48.502717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce87de-5b8e-4d65-b857-d91053eb9206 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:48.507217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cc67b-eb2a-47df-b71d-7c21408fcd8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:48.507959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cc67b-eb2a-47df-b71d-7c21408fcd8d service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.508321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cc67b-eb2a-47df-b71d-7c21408fcd8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.508671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cc67b-eb2a-47df-b71d-7c21408fcd8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.523002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6cc67b-eb2a-47df-b71d-7c21408fcd8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:48.524508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6cc67b-eb2a-47df-b71d-7c21408fcd8d service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:48.537444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 914/1804] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:28:48.549042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 891/1805] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:48 2022] PUT /identity/v3/projects/0948b03ac360454490482c4a7ef7c7ae/users/1d1f0a3c098f48c2a17d01a7a10129e4/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:28:48.553548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cc67b-eb2a-47df-b71d-7c21408fcd8d service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:48.570946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cc67b-eb2a-47df-b71d-7c21408fcd8d service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:48.573212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d84117-8add-4685-a3c1-3891505e74b6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:48.660641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141976-f273-40a9-b139-e1b0ff03f1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:48.661576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141976-f273-40a9-b139-e1b0ff03f1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.669146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141976-f273-40a9-b139-e1b0ff03f1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.669146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141976-f273-40a9-b139-e1b0ff03f1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.669146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141976-f273-40a9-b139-e1b0ff03f1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:48.669146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141976-f273-40a9-b139-e1b0ff03f1ba admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:48.681099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d84117-8add-4685-a3c1-3891505e74b6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:48.683065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 915/1806] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:48.686880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:48.687121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141976-f273-40a9-b139-e1b0ff03f1ba admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:48.687550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.687841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.688183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] PATH_INFO: `/v3/users/9c065f5c6b61406f99cb3b0aa86d0974` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.691337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c065f5c6b61406f99cb3b0aa86d0974)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:48.694950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:48.700569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141976-f273-40a9-b139-e1b0ff03f1ba admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:48.706857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afff45-2bfd-4191-9865-3e6533ef3935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:48.707549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afff45-2bfd-4191-9865-3e6533ef3935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.707839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afff45-2bfd-4191-9865-3e6533ef3935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.708122 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afff45-2bfd-4191-9865-3e6533ef3935 admin admin] PATH_INFO: `/v3/projects/0948b03ac360454490482c4a7ef7c7ae/users/1d1f0a3c098f48c2a17d01a7a10129e4/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.710804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afff45-2bfd-4191-9865-3e6533ef3935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0948b03ac360454490482c4a7ef7c7ae, user_id=1d1f0a3c098f48c2a17d01a7a10129e4, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:48.714246 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afff45-2bfd-4191-9865-3e6533ef3935 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:48.733348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 916/1807] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:48 2022] PUT /identity/v3/projects/0948b03ac360454490482c4a7ef7c7ae/users/1d1f0a3c098f48c2a17d01a7a10129e4/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:48.744327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3a26f-5b43-40f6-a017-42b8af75872b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.744899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3a26f-5b43-40f6-a017-42b8af75872b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.745186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3a26f-5b43-40f6-a017-42b8af75872b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.776908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-0da3a26f-5b43-40f6-a017-42b8af75872b None None] MFA Rules not processed for user `1d1f0a3c098f48c2a17d01a7a10129e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:48.778223 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0da3a26f-5b43-40f6-a017-42b8af75872b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:48.824690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:48.877383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 917/1808] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:48.888772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3a26f-5b43-40f6-a017-42b8af75872b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:48.961595 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c065f5c6b61406f99cb3b0aa86d0974'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:48.989354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3a26f-5b43-40f6-a017-42b8af75872b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:48.994258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e52b7-9d12-452d-811a-ab213508b8b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:48.994910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e52b7-9d12-452d-811a-ab213508b8b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:48.995281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e52b7-9d12-452d-811a-ab213508b8b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:48.995463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e52b7-9d12-452d-811a-ab213508b8b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:48.997906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c065f5c6b61406f99cb3b0aa86d0974'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:49.022708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:49.025096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c065f5c6b61406f99cb3b0aa86d0974'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:49.077718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e52b7-9d12-452d-811a-ab213508b8b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:49.079950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e52b7-9d12-452d-811a-ab213508b8b6 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:49.090387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 918/1809] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:28:49.119723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c065f5c6b61406f99cb3b0aa86d0974'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:49.120521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c065f5c6b61406f99cb3b0aa86d0974'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:49.136234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 892/1810] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:48 2022] DELETE /identity/v3/users/9c065f5c6b61406f99cb3b0aa86d0974 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:49.136321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b190f9-6ac5-4a48-86ad-05b61a0a7e2c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:49.136321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b190f9-6ac5-4a48-86ad-05b61a0a7e2c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:49.136321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b190f9-6ac5-4a48-86ad-05b61a0a7e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:49.173123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f8b190f9-6ac5-4a48-86ad-05b61a0a7e2c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:49.174840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f8b190f9-6ac5-4a48-86ad-05b61a0a7e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:49.285786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 919/1811] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:49.292437 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:49.294182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:49.353462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:49.355158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be176148-d1c4-43be-a2ac-090bf4715c0a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:49.358854 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a33a27-7186-4ea0-8061-21c19229e359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:49.363301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a33a27-7186-4ea0-8061-21c19229e359 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:49.363301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a33a27-7186-4ea0-8061-21c19229e359 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:49.363301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a33a27-7186-4ea0-8061-21c19229e359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:49.371679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a33a27-7186-4ea0-8061-21c19229e359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:49.373137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a33a27-7186-4ea0-8061-21c19229e359 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:49.388960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 893/1812] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:49.582720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:49.857722 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b190f9-6ac5-4a48-86ad-05b61a0a7e2c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:49.861202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f8b190f9-6ac5-4a48-86ad-05b61a0a7e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:49.885615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b190f9-6ac5-4a48-86ad-05b61a0a7e2c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:49.892152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:49.893083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:49.893611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:49.894540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] PATH_INFO: `/v3/projects/81a3fce7cb1447e3979d9a2313f54dd2` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:49.895621 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a3fce7cb1447e3979d9a2313f54dd2)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:49.900172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:49.995033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:50.103339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a3fce7cb1447e3979d9a2313f54dd2', 'request_id': 'req-41ecd743-a491-4512-886c-455a56e0ba66'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:50.113483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a3fce7cb1447e3979d9a2313f54dd2', 'request_id': 'req-41ecd743-a491-4512-886c-455a56e0ba66'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:50.243046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a3fce7cb1447e3979d9a2313f54dd2', 'request_id': 'req-41ecd743-a491-4512-886c-455a56e0ba66'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:50.257099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] The token cache is being invalidate because project 81a3fce7cb1447e3979d9a2313f54dd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:50.260244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:50.260244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:50.260244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 920/1813] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:49 2022] DELETE /identity/v3/projects/81a3fce7cb1447e3979d9a2313f54dd2 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:50.264742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a33a27-7186-4ea0-8061-21c19229e359 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:50.276566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a33a27-7186-4ea0-8061-21c19229e359 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:50.340562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:50.341331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:50.341684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:50.342977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] PATH_INFO: `/v3/users/825658bff3e747fca86db2b3862f020b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:50.345596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=825658bff3e747fca86db2b3862f020b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:50.348064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:50.572875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '825658bff3e747fca86db2b3862f020b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:50.585258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '825658bff3e747fca86db2b3862f020b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:50.588071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:50.593042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '825658bff3e747fca86db2b3862f020b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:50.656392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '825658bff3e747fca86db2b3862f020b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:50.657563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '825658bff3e747fca86db2b3862f020b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:50.664357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 894/1814] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:50 2022] DELETE /identity/v3/users/825658bff3e747fca86db2b3862f020b => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:51.095108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:51.096045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:51.111483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecd743-a491-4512-886c-455a56e0ba66 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:51.195042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:51.195683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:51.196038 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:51.196398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] PATH_INFO: `/v3/projects/e647475285ff45419de2a25cb0cb482c` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:51.197702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e647475285ff45419de2a25cb0cb482c)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:51.200669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:51.265387 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:51.267792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:51.292099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:51.328061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e647475285ff45419de2a25cb0cb482c', 'request_id': 'req-0566a6ae-9f7d-4af6-952b-3af05b8044f2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:51.337515 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef8402-e36e-4c96-8059-b98771c7c312 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:51.337944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e647475285ff45419de2a25cb0cb482c', 'request_id': 'req-0566a6ae-9f7d-4af6-952b-3af05b8044f2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:51.403372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943a81e-1713-4445-853d-20d842b47de4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:51.404468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a81e-1713-4445-853d-20d842b47de4 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:51.404757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a81e-1713-4445-853d-20d842b47de4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:51.405045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a81e-1713-4445-853d-20d842b47de4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:51.408154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6943a81e-1713-4445-853d-20d842b47de4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:51.462401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e647475285ff45419de2a25cb0cb482c', 'request_id': 'req-0566a6ae-9f7d-4af6-952b-3af05b8044f2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:51.480240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:51.482779 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] The token cache is being invalidate because project e647475285ff45419de2a25cb0cb482c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:51.483088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:51.489130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0566a6ae-9f7d-4af6-952b-3af05b8044f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:51.492000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 921/1815] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:51 2022] DELETE /identity/v3/projects/e647475285ff45419de2a25cb0cb482c => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:51.497680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d57e8-9e60-42e8-82e7-0b37f7b1a639 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:51.497980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d57e8-9e60-42e8-82e7-0b37f7b1a639 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:51.498282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d57e8-9e60-42e8-82e7-0b37f7b1a639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:51.499673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943a81e-1713-4445-853d-20d842b47de4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:51.501540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943a81e-1713-4445-853d-20d842b47de4 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:51.512485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 895/1816] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:28:51.537500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-421d57e8-9e60-42e8-82e7-0b37f7b1a639 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:51.538833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-421d57e8-9e60-42e8-82e7-0b37f7b1a639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:51.628581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 922/1817] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:51.630778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd86c-9731-43fc-aee9-1936d326d774 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:51.631085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd86c-9731-43fc-aee9-1936d326d774 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:51.631376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd86c-9731-43fc-aee9-1936d326d774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:51.677940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f25bd86c-9731-43fc-aee9-1936d326d774 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:51.679198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f25bd86c-9731-43fc-aee9-1936d326d774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:51.705520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 896/1818] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:51.711585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d57e8-9e60-42e8-82e7-0b37f7b1a639 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:51.727044 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d57e8-9e60-42e8-82e7-0b37f7b1a639 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:51.733025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2f48c-05d3-4c69-89a8-c401755a1950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:51.733724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2f48c-05d3-4c69-89a8-c401755a1950 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:51.734011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2f48c-05d3-4c69-89a8-c401755a1950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:51.734293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2f48c-05d3-4c69-89a8-c401755a1950 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:51.735113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2f48c-05d3-4c69-89a8-c401755a1950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:51.735678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2f48c-05d3-4c69-89a8-c401755a1950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:51.737533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2f48c-05d3-4c69-89a8-c401755a1950 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:51.758387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 923/1819] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:51.763723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd86c-9731-43fc-aee9-1936d326d774 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:51.777880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd86c-9731-43fc-aee9-1936d326d774 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:51.784403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dc83e-8330-4dbc-b3d7-15ad2d08536f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:51.785137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc83e-8330-4dbc-b3d7-15ad2d08536f admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:51.785527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc83e-8330-4dbc-b3d7-15ad2d08536f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:51.786034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc83e-8330-4dbc-b3d7-15ad2d08536f admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:51.786936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dc83e-8330-4dbc-b3d7-15ad2d08536f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:51.789366 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dc83e-8330-4dbc-b3d7-15ad2d08536f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:51.909357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 897/1820] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:51 2022] POST /identity/v3/projects => generated 389 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:51.913190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2f48c-05d3-4c69-89a8-c401755a1950 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:51.925352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2f48c-05d3-4c69-89a8-c401755a1950 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:52.019923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6683cef-8a46-4fad-ad97-bb9575dfdb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:52.020784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6683cef-8a46-4fad-ad97-bb9575dfdb04 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:52.021175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6683cef-8a46-4fad-ad97-bb9575dfdb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:52.021635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6683cef-8a46-4fad-ad97-bb9575dfdb04 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:52.023077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6683cef-8a46-4fad-ad97-bb9575dfdb04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:52.025500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6683cef-8a46-4fad-ad97-bb9575dfdb04 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:52.967163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 924/1821] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:51 2022] POST /identity/v3/users => generated 295 bytes in 1054 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:28:52.977363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dc83e-8330-4dbc-b3d7-15ad2d08536f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:52.988257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dc83e-8330-4dbc-b3d7-15ad2d08536f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:52.993582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409ccb8-c822-440f-ab45-d9c6f091a599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:52.994555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409ccb8-c822-440f-ab45-d9c6f091a599 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:52.994650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409ccb8-c822-440f-ab45-d9c6f091a599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:52.995031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409ccb8-c822-440f-ab45-d9c6f091a599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:52.995831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409ccb8-c822-440f-ab45-d9c6f091a599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:52.997504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409ccb8-c822-440f-ab45-d9c6f091a599 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:53.008353 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 898/1822] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:53.012532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6683cef-8a46-4fad-ad97-bb9575dfdb04 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:53.023100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6683cef-8a46-4fad-ad97-bb9575dfdb04 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:53.027731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86d35a-be89-4f47-96de-5a014307b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:53.028355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86d35a-be89-4f47-96de-5a014307b99a admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:53.028639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86d35a-be89-4f47-96de-5a014307b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:53.028985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86d35a-be89-4f47-96de-5a014307b99a admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:53.029723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86d35a-be89-4f47-96de-5a014307b99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:53.031352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86d35a-be89-4f47-96de-5a014307b99a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:53.046684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 925/1823] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:53.047856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409ccb8-c822-440f-ab45-d9c6f091a599 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:53.060089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409ccb8-c822-440f-ab45-d9c6f091a599 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:53.068259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc7c72-7a91-40a3-b4d9-7b207530ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:53.068961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc7c72-7a91-40a3-b4d9-7b207530ec45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:53.072749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc7c72-7a91-40a3-b4d9-7b207530ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:53.072749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc7c72-7a91-40a3-b4d9-7b207530ec45 admin admin] PATH_INFO: `/v3/projects/2511253fb8964ddba720432e179b7617/users/ff6043a1f36540bb8fcc477b5c447021/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:53.086489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc7c72-7a91-40a3-b4d9-7b207530ec45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2511253fb8964ddba720432e179b7617, user_id=ff6043a1f36540bb8fcc477b5c447021, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:53.089415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc7c72-7a91-40a3-b4d9-7b207530ec45 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:53.199988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 899/1824] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:53 2022] PUT /identity/v3/projects/2511253fb8964ddba720432e179b7617/users/ff6043a1f36540bb8fcc477b5c447021/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:53.211434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b70b8a-70b4-42f2-8eb3-2af165a66261 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:53.212254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b70b8a-70b4-42f2-8eb3-2af165a66261 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:53.212902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b70b8a-70b4-42f2-8eb3-2af165a66261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:53.246182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-10b70b8a-70b4-42f2-8eb3-2af165a66261 None None] MFA Rules not processed for user `ff6043a1f36540bb8fcc477b5c447021`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:53.247459 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-10b70b8a-70b4-42f2-8eb3-2af165a66261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:53.306721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:53.317104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 926/1825] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:53.321587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc7c72-7a91-40a3-b4d9-7b207530ec45 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:53.355579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:53.394815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc7c72-7a91-40a3-b4d9-7b207530ec45 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:53.401765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99afc40-c671-4754-b2ac-f1ec1a7503e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:53.402666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99afc40-c671-4754-b2ac-f1ec1a7503e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:53.403083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99afc40-c671-4754-b2ac-f1ec1a7503e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:53.403495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99afc40-c671-4754-b2ac-f1ec1a7503e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:53.404522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99afc40-c671-4754-b2ac-f1ec1a7503e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:53.408492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99afc40-c671-4754-b2ac-f1ec1a7503e2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:53.528148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 900/1826] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:53 2022] POST /identity/v3/projects => generated 389 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:28:53.530140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b70b8a-70b4-42f2-8eb3-2af165a66261 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:53.545553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b70b8a-70b4-42f2-8eb3-2af165a66261 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:53.628281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c6254-7cd5-4f45-9c75-a901cff6ca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:53.629090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c6254-7cd5-4f45-9c75-a901cff6ca1b admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:53.631415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c6254-7cd5-4f45-9c75-a901cff6ca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:53.631415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c6254-7cd5-4f45-9c75-a901cff6ca1b admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:53.631415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c6254-7cd5-4f45-9c75-a901cff6ca1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:53.632805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c6254-7cd5-4f45-9c75-a901cff6ca1b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:53.742640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99afc40-c671-4754-b2ac-f1ec1a7503e2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:53.751063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 927/1827] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:53 2022] POST /identity/v3/users => generated 301 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:28:53.757716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99afc40-c671-4754-b2ac-f1ec1a7503e2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:53.768617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba3cf-033f-4827-ae4f-949190465ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:53.769525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba3cf-033f-4827-ae4f-949190465ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:53.770780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba3cf-033f-4827-ae4f-949190465ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:53.771257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba3cf-033f-4827-ae4f-949190465ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:53.772100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba3cf-033f-4827-ae4f-949190465ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:53.773922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba3cf-033f-4827-ae4f-949190465ec8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:53.807795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 901/1828] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:53.815076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c6254-7cd5-4f45-9c75-a901cff6ca1b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:53.827609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c6254-7cd5-4f45-9c75-a901cff6ca1b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:53.835566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61005c6e-b87f-40ce-89cb-a4d21ecddf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:53.835566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61005c6e-b87f-40ce-89cb-a4d21ecddf10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:53.836427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61005c6e-b87f-40ce-89cb-a4d21ecddf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:53.836724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61005c6e-b87f-40ce-89cb-a4d21ecddf10 admin admin] PATH_INFO: `/v3/projects/3101ff57a2044428b062046408ca4665/users/02cc05daa7b64378a06e6ae7c7ee8323/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:53.856010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61005c6e-b87f-40ce-89cb-a4d21ecddf10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3101ff57a2044428b062046408ca4665, user_id=02cc05daa7b64378a06e6ae7c7ee8323, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:53.859563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61005c6e-b87f-40ce-89cb-a4d21ecddf10 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:54.029867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 928/1829] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:53 2022] PUT /identity/v3/projects/3101ff57a2044428b062046408ca4665/users/02cc05daa7b64378a06e6ae7c7ee8323/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:54.034814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba3cf-033f-4827-ae4f-949190465ec8 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:54.045575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba3cf-033f-4827-ae4f-949190465ec8 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:54.129651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb0064-4c2e-4e10-82b0-658f3a76cee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:54.131713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb0064-4c2e-4e10-82b0-658f3a76cee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:54.132180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb0064-4c2e-4e10-82b0-658f3a76cee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:54.132453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb0064-4c2e-4e10-82b0-658f3a76cee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:54.133420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb0064-4c2e-4e10-82b0-658f3a76cee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:54.135360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb0064-4c2e-4e10-82b0-658f3a76cee4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:54.153181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 902/1830] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:54.168371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61005c6e-b87f-40ce-89cb-a4d21ecddf10 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:54.170886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61005c6e-b87f-40ce-89cb-a4d21ecddf10 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:54.177579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebc8a5-bf56-4aa1-8b93-024350382eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:54.178282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebc8a5-bf56-4aa1-8b93-024350382eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:54.178574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebc8a5-bf56-4aa1-8b93-024350382eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:54.178859 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebc8a5-bf56-4aa1-8b93-024350382eac admin admin] PATH_INFO: `/v3/projects/3101ff57a2044428b062046408ca4665/users/02cc05daa7b64378a06e6ae7c7ee8323/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:54.180901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebc8a5-bf56-4aa1-8b93-024350382eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=3101ff57a2044428b062046408ca4665, user_id=02cc05daa7b64378a06e6ae7c7ee8323, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:54.183581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebc8a5-bf56-4aa1-8b93-024350382eac admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:54.195207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 929/1831] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:28:54 2022] PUT /identity/v3/projects/3101ff57a2044428b062046408ca4665/users/02cc05daa7b64378a06e6ae7c7ee8323/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:54.206358 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d5bdc-8610-4ae5-93ba-3dd92c48e409 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:54.214863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d5bdc-8610-4ae5-93ba-3dd92c48e409 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:54.214863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d5bdc-8610-4ae5-93ba-3dd92c48e409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:54.258849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5b0d5bdc-8610-4ae5-93ba-3dd92c48e409 None None] MFA Rules not processed for user `02cc05daa7b64378a06e6ae7c7ee8323`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:54.271150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5b0d5bdc-8610-4ae5-93ba-3dd92c48e409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:54.371602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 903/1832] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:54.383533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebc8a5-bf56-4aa1-8b93-024350382eac admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:54.385777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c1ebc8a5-bf56-4aa1-8b93-024350382eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:54.428373 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d5bdc-8610-4ae5-93ba-3dd92c48e409 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:54.430140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5b0d5bdc-8610-4ae5-93ba-3dd92c48e409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:54.487073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebc8a5-bf56-4aa1-8b93-024350382eac admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:54.492176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeed694-b446-4a92-8f4d-51d98041b0e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:54.493701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeed694-b446-4a92-8f4d-51d98041b0e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:54.493701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeed694-b446-4a92-8f4d-51d98041b0e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:54.494260 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeed694-b446-4a92-8f4d-51d98041b0e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:54.508801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeed694-b446-4a92-8f4d-51d98041b0e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:54.514627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeed694-b446-4a92-8f4d-51d98041b0e5 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:54.533294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 930/1833] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:54.565231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:54.577632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d5bdc-8610-4ae5-93ba-3dd92c48e409 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:54.588615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f82bd-26ec-4232-a357-88975591c790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:54.591333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f82bd-26ec-4232-a357-88975591c790 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:54.591333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f82bd-26ec-4232-a357-88975591c790 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:54.591333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f82bd-26ec-4232-a357-88975591c790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:54.607804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2e5f82bd-26ec-4232-a357-88975591c790 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:54.776172 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f82bd-26ec-4232-a357-88975591c790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:54.777993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f82bd-26ec-4232-a357-88975591c790 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:54.796826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeed694-b446-4a92-8f4d-51d98041b0e5 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:54.817625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 904/1834] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:54.952903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:54.980214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeed694-b446-4a92-8f4d-51d98041b0e5 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:54.988747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:54.990228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:54.990520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:54.990808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] PATH_INFO: `/v3/users/ff6043a1f36540bb8fcc477b5c447021` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:54.992047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6043a1f36540bb8fcc477b5c447021)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:54.994259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:55.293333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6043a1f36540bb8fcc477b5c447021'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:55.308252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085e6c4-90b5-4685-8223-afb1fb9e0f54 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:55.309506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085e6c4-90b5-4685-8223-afb1fb9e0f54 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:55.309959 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085e6c4-90b5-4685-8223-afb1fb9e0f54 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:55.314058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 905/1835] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:28:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:55.327062 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6043a1f36540bb8fcc477b5c447021'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:55.327844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6043a1f36540bb8fcc477b5c447021'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:55.333401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e70f1-16ae-4001-be3e-f5e129f2ee0c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:55.333401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e70f1-16ae-4001-be3e-f5e129f2ee0c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:55.333401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e70f1-16ae-4001-be3e-f5e129f2ee0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:55.340333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6043a1f36540bb8fcc477b5c447021'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:55.384848 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-553e70f1-16ae-4001-be3e-f5e129f2ee0c None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:55.386145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-553e70f1-16ae-4001-be3e-f5e129f2ee0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:55.464125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b36df530-8833-462a-acc1-1913d841ab4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6043a1f36540bb8fcc477b5c447021'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:55.478719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 931/1836] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:54 2022] DELETE /identity/v3/users/ff6043a1f36540bb8fcc477b5c447021 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:28:55.483576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6230d3-3032-4386-b0b9-f5acdcb37467 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:55.483940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6230d3-3032-4386-b0b9-f5acdcb37467 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:55.484222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6230d3-3032-4386-b0b9-f5acdcb37467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:55.520947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 906/1837] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:28:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:55.533195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-dc6230d3-3032-4386-b0b9-f5acdcb37467 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:55.534417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-dc6230d3-3032-4386-b0b9-f5acdcb37467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:55.631734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 932/1838] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:55.643359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e70f1-16ae-4001-be3e-f5e129f2ee0c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:55.644933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-553e70f1-16ae-4001-be3e-f5e129f2ee0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:55.758232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e70f1-16ae-4001-be3e-f5e129f2ee0c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:55.781177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:55.833530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6da5c-60b3-431a-bc79-fed8443d85b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:55.835027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6da5c-60b3-431a-bc79-fed8443d85b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:55.835541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6da5c-60b3-431a-bc79-fed8443d85b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:55.836311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6da5c-60b3-431a-bc79-fed8443d85b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:55.853423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6da5c-60b3-431a-bc79-fed8443d85b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:55.855082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6da5c-60b3-431a-bc79-fed8443d85b8 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:55.872236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 907/1839] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:28:56.030602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6230d3-3032-4386-b0b9-f5acdcb37467 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:56.034023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-dc6230d3-3032-4386-b0b9-f5acdcb37467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:56.169363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6230d3-3032-4386-b0b9-f5acdcb37467 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:56.183353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f89e8-9dff-4155-bf43-caf2bda0dbe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:56.184438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f89e8-9dff-4155-bf43-caf2bda0dbe6 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:56.184908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f89e8-9dff-4155-bf43-caf2bda0dbe6 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:56.185408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f89e8-9dff-4155-bf43-caf2bda0dbe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:56.193578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:56.197942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f89e8-9dff-4155-bf43-caf2bda0dbe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:56.199852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f89e8-9dff-4155-bf43-caf2bda0dbe6 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:56.215055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 933/1840] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:56.893658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6da5c-60b3-431a-bc79-fed8443d85b8 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:56.894995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1ff6da5c-60b3-431a-bc79-fed8443d85b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:56.915770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6da5c-60b3-431a-bc79-fed8443d85b8 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:56.928586 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:56.929489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:56.929943 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:56.930405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] PATH_INFO: `/v3/projects/2511253fb8964ddba720432e179b7617` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:56.933932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2511253fb8964ddba720432e179b7617)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:56.936430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:57.182963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2511253fb8964ddba720432e179b7617', 'request_id': 'req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:57.200013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2511253fb8964ddba720432e179b7617', 'request_id': 'req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:57.229286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2511253fb8964ddba720432e179b7617', 'request_id': 'req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:57.386315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] The token cache is being invalidate because project 2511253fb8964ddba720432e179b7617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:57.386632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:57.389662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:57.392998 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 908/1841] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:56 2022] DELETE /identity/v3/projects/2511253fb8964ddba720432e179b7617 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:57.398125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f89e8-9dff-4155-bf43-caf2bda0dbe6 service nova] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:57.417600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f89e8-9dff-4155-bf43-caf2bda0dbe6 service nova] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:57.562798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:57.563672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:57.567042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:57.567619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] PATH_INFO: `/v3/users/02cc05daa7b64378a06e6ae7c7ee8323` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:57.570055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02cc05daa7b64378a06e6ae7c7ee8323)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:57.574698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:57.866131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02cc05daa7b64378a06e6ae7c7ee8323'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:57.889045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02cc05daa7b64378a06e6ae7c7ee8323'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:57.891625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02cc05daa7b64378a06e6ae7c7ee8323'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:57.901029 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02cc05daa7b64378a06e6ae7c7ee8323'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:58.016496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6b253b88-cc64-405e-ba55-2b1bd633cc31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02cc05daa7b64378a06e6ae7c7ee8323'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:58.026945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 934/1842] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:28:57 2022] DELETE /identity/v3/users/02cc05daa7b64378a06e6ae7c7ee8323 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:58.561489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:58.564552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:58.589144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753ae9c-537a-4dd4-b3bc-aad1ac7fd797 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:58.761601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:58.762266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:58.762556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:58.762837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] PATH_INFO: `/v3/projects/3101ff57a2044428b062046408ca4665` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:58.765490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3101ff57a2044428b062046408ca4665)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:58.769910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:59.007344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:59.010129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3101ff57a2044428b062046408ca4665', 'request_id': 'req-5e3314fe-4727-47d8-8fd6-05842cc7abba'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:59.017922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3101ff57a2044428b062046408ca4665', 'request_id': 'req-5e3314fe-4727-47d8-8fd6-05842cc7abba'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:59.035319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3101ff57a2044428b062046408ca4665', 'request_id': 'req-5e3314fe-4727-47d8-8fd6-05842cc7abba'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:59.147175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] The token cache is being invalidate because project 3101ff57a2044428b062046408ca4665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:28:59.147475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:59.148474 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5e3314fe-4727-47d8-8fd6-05842cc7abba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:28:59.150313 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 909/1843] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:28:58 2022] DELETE /identity/v3/projects/3101ff57a2044428b062046408ca4665 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:28:59.155713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fbd31-6047-4967-91cc-b00e2db5765e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:59.156292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fbd31-6047-4967-91cc-b00e2db5765e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:59.156292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fbd31-6047-4967-91cc-b00e2db5765e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:59.181046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-a02fbd31-6047-4967-91cc-b00e2db5765e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:59.182507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a02fbd31-6047-4967-91cc-b00e2db5765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:59.307196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 935/1844] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:59.315794 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdabb6-c44e-445f-8de8-e603d8c69194 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:59.316113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdabb6-c44e-445f-8de8-e603d8c69194 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:59.316367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdabb6-c44e-445f-8de8-e603d8c69194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:59.386333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-83bdabb6-c44e-445f-8de8-e603d8c69194 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:59.390659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-83bdabb6-c44e-445f-8de8-e603d8c69194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:59.402499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 910/1845] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:59.407033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fbd31-6047-4967-91cc-b00e2db5765e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:59.433862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fbd31-6047-4967-91cc-b00e2db5765e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:59.440236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3860f6-bad0-4efe-a10b-5d2a98afbe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:59.441251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3860f6-bad0-4efe-a10b-5d2a98afbe34 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:59.441744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3860f6-bad0-4efe-a10b-5d2a98afbe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:59.442184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3860f6-bad0-4efe-a10b-5d2a98afbe34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:59.443960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3860f6-bad0-4efe-a10b-5d2a98afbe34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:28:59.444496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3860f6-bad0-4efe-a10b-5d2a98afbe34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:59.446616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3860f6-bad0-4efe-a10b-5d2a98afbe34 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:59.472495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:59.476925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 936/1846] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:28:59.481066 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdabb6-c44e-445f-8de8-e603d8c69194 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:59.494771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdabb6-c44e-445f-8de8-e603d8c69194 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:59.502140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70930198-6a0d-426c-a637-e95e2e6d7e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:59.502893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70930198-6a0d-426c-a637-e95e2e6d7e30 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:59.504804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70930198-6a0d-426c-a637-e95e2e6d7e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:59.504804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70930198-6a0d-426c-a637-e95e2e6d7e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:59.504804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70930198-6a0d-426c-a637-e95e2e6d7e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:59.507552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70930198-6a0d-426c-a637-e95e2e6d7e30 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:59.660313 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 911/1847] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:28:59 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:28:59.664745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3860f6-bad0-4efe-a10b-5d2a98afbe34 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:59.679123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3860f6-bad0-4efe-a10b-5d2a98afbe34 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:59.753548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ffe11-03a1-4997-8f25-9650d07df369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:59.754369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ffe11-03a1-4997-8f25-9650d07df369 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:59.755566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ffe11-03a1-4997-8f25-9650d07df369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:59.755998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ffe11-03a1-4997-8f25-9650d07df369 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:59.757108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ffe11-03a1-4997-8f25-9650d07df369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:59.759413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ffe11-03a1-4997-8f25-9650d07df369 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:59.914408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 937/1848] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:28:59 2022] POST /identity/v3/users => generated 287 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:28:59.916108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70930198-6a0d-426c-a637-e95e2e6d7e30 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:59.928834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70930198-6a0d-426c-a637-e95e2e6d7e30 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:59.934375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3cf0c-f71f-4fa5-93b8-1a4a19792b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:59.935399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3cf0c-f71f-4fa5-93b8-1a4a19792b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:59.935844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3cf0c-f71f-4fa5-93b8-1a4a19792b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:59.936275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3cf0c-f71f-4fa5-93b8-1a4a19792b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:59.937155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3cf0c-f71f-4fa5-93b8-1a4a19792b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:59.939704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3cf0c-f71f-4fa5-93b8-1a4a19792b4b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:59.951342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 912/1849] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:28:59.956103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ffe11-03a1-4997-8f25-9650d07df369 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:59.971661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ffe11-03a1-4997-8f25-9650d07df369 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:59.977236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3a207-d182-4de2-9de4-aa91afd76486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:59.977847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3a207-d182-4de2-9de4-aa91afd76486 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:59.978142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3a207-d182-4de2-9de4-aa91afd76486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:59.978425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3a207-d182-4de2-9de4-aa91afd76486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:59.979136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3a207-d182-4de2-9de4-aa91afd76486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:59.981168 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3a207-d182-4de2-9de4-aa91afd76486 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:59.998235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 938/1850] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:00.003332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3cf0c-f71f-4fa5-93b8-1a4a19792b4b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:00.024754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3cf0c-f71f-4fa5-93b8-1a4a19792b4b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:00.031962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09348722-9ac9-47e9-9ced-f9dd14ee423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:00.032881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09348722-9ac9-47e9-9ced-f9dd14ee423c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:00.033261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09348722-9ac9-47e9-9ced-f9dd14ee423c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:00.033649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09348722-9ac9-47e9-9ced-f9dd14ee423c admin admin] PATH_INFO: `/v3/projects/22a64b6b084848ec89599880170c719e/users/eb2a7c1097e0421492e8460e9b64a27b/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:00.057186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09348722-9ac9-47e9-9ced-f9dd14ee423c admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a64b6b084848ec89599880170c719e, user_id=eb2a7c1097e0421492e8460e9b64a27b, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:00.060141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09348722-9ac9-47e9-9ced-f9dd14ee423c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:00.186286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 913/1851] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:00 2022] PUT /identity/v3/projects/22a64b6b084848ec89599880170c719e/users/eb2a7c1097e0421492e8460e9b64a27b/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:00.192411 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0af89b-4dff-4ae8-ac52-8cda9ec7949c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:00.192877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0af89b-4dff-4ae8-ac52-8cda9ec7949c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:00.193045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0af89b-4dff-4ae8-ac52-8cda9ec7949c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:00.220882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-dc0af89b-4dff-4ae8-ac52-8cda9ec7949c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:00.222741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-dc0af89b-4dff-4ae8-ac52-8cda9ec7949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:00.247564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09348722-9ac9-47e9-9ced-f9dd14ee423c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:00.249535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-09348722-9ac9-47e9-9ced-f9dd14ee423c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:00.313767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 939/1852] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:00.323722 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0af89b-4dff-4ae8-ac52-8cda9ec7949c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:00.325288 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-dc0af89b-4dff-4ae8-ac52-8cda9ec7949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:00.343504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09348722-9ac9-47e9-9ced-f9dd14ee423c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:00.362883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:00.362883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:00.362883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:00.362883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] PATH_INFO: `/v3/users/ce89c4a504bb4601b6ec8c6ad32c2aa0` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:00.368021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce89c4a504bb4601b6ec8c6ad32c2aa0)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:00.371321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:00.411193 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0af89b-4dff-4ae8-ac52-8cda9ec7949c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:00.415712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834661f0-df6c-4a3a-9967-b358764ee856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:00.416395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834661f0-df6c-4a3a-9967-b358764ee856 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:00.416694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834661f0-df6c-4a3a-9967-b358764ee856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:00.417045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834661f0-df6c-4a3a-9967-b358764ee856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:00.428814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834661f0-df6c-4a3a-9967-b358764ee856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:00.430313 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834661f0-df6c-4a3a-9967-b358764ee856 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:00.442430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 940/1853] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:00.659297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce89c4a504bb4601b6ec8c6ad32c2aa0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:00.675477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce89c4a504bb4601b6ec8c6ad32c2aa0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:00.683961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce89c4a504bb4601b6ec8c6ad32c2aa0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:00.759093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce89c4a504bb4601b6ec8c6ad32c2aa0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:00.760697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce89c4a504bb4601b6ec8c6ad32c2aa0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:00.770489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 914/1854] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:00 2022] DELETE /identity/v3/users/ce89c4a504bb4601b6ec8c6ad32c2aa0 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:00.776828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b98444-9a10-4303-8ba1-3ec1cd93208d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:00.776828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b98444-9a10-4303-8ba1-3ec1cd93208d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:00.777034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b98444-9a10-4303-8ba1-3ec1cd93208d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:00.824228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-01b98444-9a10-4303-8ba1-3ec1cd93208d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:00.825778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-01b98444-9a10-4303-8ba1-3ec1cd93208d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:00.925679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 941/1855] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:00.947615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:00.947615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:01.034850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009c1d3-444f-4342-8190-a261eb47840f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:01.039969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc4255-72a2-4735-8de3-345a9aaea5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:01.040667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc4255-72a2-4735-8de3-345a9aaea5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:01.040962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc4255-72a2-4735-8de3-345a9aaea5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:01.041243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc4255-72a2-4735-8de3-345a9aaea5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:01.056598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc4255-72a2-4735-8de3-345a9aaea5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:01.061744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc4255-72a2-4735-8de3-345a9aaea5a3 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:01.080085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 915/1856] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:01.912922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b98444-9a10-4303-8ba1-3ec1cd93208d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:01.915039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-01b98444-9a10-4303-8ba1-3ec1cd93208d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:01.928765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b98444-9a10-4303-8ba1-3ec1cd93208d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:01.930730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:01.933839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:01.934759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:01.935117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:01.940112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] PATH_INFO: `/v3/projects/77ff6eefb6164ff8a16a0988f9667420` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:01.940112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ff6eefb6164ff8a16a0988f9667420)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:01.940112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:02.084945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ff6eefb6164ff8a16a0988f9667420', 'request_id': 'req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:02.092370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ff6eefb6164ff8a16a0988f9667420', 'request_id': 'req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:02.160870 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ded12-ef63-4791-9406-ee7bebc25151 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:02.161149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ded12-ef63-4791-9406-ee7bebc25151 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:02.161460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ded12-ef63-4791-9406-ee7bebc25151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:02.175737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ff6eefb6164ff8a16a0988f9667420', 'request_id': 'req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:02.205407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] The token cache is being invalidate because project 77ff6eefb6164ff8a16a0988f9667420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:02.205970 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:02.206823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:02.207949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a53ded12-ef63-4791-9406-ee7bebc25151 None None] MFA Rules not processed for user `eb2a7c1097e0421492e8460e9b64a27b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:02.208611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 942/1857] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:01 2022] DELETE /identity/v3/projects/77ff6eefb6164ff8a16a0988f9667420 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:02.209938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a53ded12-ef63-4791-9406-ee7bebc25151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:02.215876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:02.217879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:02.294220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 916/1858] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:02.306932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ded12-ef63-4791-9406-ee7bebc25151 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:02.310650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a53ded12-ef63-4791-9406-ee7bebc25151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:02.310754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9c231-e56c-4c57-9d3d-9e86f1694e3a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:02.315491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:02.316028 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:02.316337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:02.316626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] PATH_INFO: `/v3/users/7d88710b8e38417ab8385bc511dec253` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:02.317960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d88710b8e38417ab8385bc511dec253)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:02.320067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:02.369185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ded12-ef63-4791-9406-ee7bebc25151 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:02.373022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c8ba1-4fc0-439d-aea6-73a0ccba3beb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:02.373664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c8ba1-4fc0-439d-aea6-73a0ccba3beb service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:02.377681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c8ba1-4fc0-439d-aea6-73a0ccba3beb service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:02.377681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c8ba1-4fc0-439d-aea6-73a0ccba3beb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:02.384357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c8ba1-4fc0-439d-aea6-73a0ccba3beb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:02.385819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c8ba1-4fc0-439d-aea6-73a0ccba3beb service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:02.395827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 917/1859] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:02.559706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d88710b8e38417ab8385bc511dec253'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:02.572847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d88710b8e38417ab8385bc511dec253'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:02.573645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d88710b8e38417ab8385bc511dec253'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:02.581287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d88710b8e38417ab8385bc511dec253'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:02.678675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d88710b8e38417ab8385bc511dec253'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:02.685513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 943/1860] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:02 2022] DELETE /identity/v3/users/7d88710b8e38417ab8385bc511dec253 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:02.936163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:03.525596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c8ba1-4fc0-439d-aea6-73a0ccba3beb service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:03.540391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c8ba1-4fc0-439d-aea6-73a0ccba3beb service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:03.600407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:03.601104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:03.601425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:03.601764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] PATH_INFO: `/v3/projects/8a3320422a6c458894aa127593f8eb29` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:03.602873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a3320422a6c458894aa127593f8eb29)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:03.605434 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:03.764408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a3320422a6c458894aa127593f8eb29', 'request_id': 'req-bd95983b-710d-4213-81d7-f0fc312ea5ca'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:03.770011 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a3320422a6c458894aa127593f8eb29', 'request_id': 'req-bd95983b-710d-4213-81d7-f0fc312ea5ca'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:03.776503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a3320422a6c458894aa127593f8eb29', 'request_id': 'req-bd95983b-710d-4213-81d7-f0fc312ea5ca'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:03.846524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] The token cache is being invalidate because project 8a3320422a6c458894aa127593f8eb29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:03.847089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:03.847822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:03.849591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 918/1861] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:03 2022] DELETE /identity/v3/projects/8a3320422a6c458894aa127593f8eb29 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:03.853591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:03.855144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:03.869739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0363b1-042c-418c-bcee-9a5b025622fc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:03.934093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:03.934762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:03.935051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:03.935454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] PATH_INFO: `/v3/users/1d1f0a3c098f48c2a17d01a7a10129e4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:03.936528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d1f0a3c098f48c2a17d01a7a10129e4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:03.939261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:04.143997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d1f0a3c098f48c2a17d01a7a10129e4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:04.155654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d1f0a3c098f48c2a17d01a7a10129e4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:04.157109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d1f0a3c098f48c2a17d01a7a10129e4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:04.164255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d1f0a3c098f48c2a17d01a7a10129e4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:04.232999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-a203179e-41b4-4555-8a37-2d6321664bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d1f0a3c098f48c2a17d01a7a10129e4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:04.239974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 944/1862] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:03 2022] DELETE /identity/v3/users/1d1f0a3c098f48c2a17d01a7a10129e4 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:04.518603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:04.770658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:04.772489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:04.784949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:04.792133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95983b-710d-4213-81d7-f0fc312ea5ca admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:04.927993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:04.928979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:04.929494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:04.929920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] PATH_INFO: `/v3/projects/0948b03ac360454490482c4a7ef7c7ae` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:04.931468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0948b03ac360454490482c4a7ef7c7ae)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:04.934673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:05.113750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0948b03ac360454490482c4a7ef7c7ae', 'request_id': 'req-94c06e77-dcc1-45d0-a6dd-4fa89911929f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:05.119964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0948b03ac360454490482c4a7ef7c7ae', 'request_id': 'req-94c06e77-dcc1-45d0-a6dd-4fa89911929f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:05.128493 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0948b03ac360454490482c4a7ef7c7ae', 'request_id': 'req-94c06e77-dcc1-45d0-a6dd-4fa89911929f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:05.260677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] The token cache is being invalidate because project 0948b03ac360454490482c4a7ef7c7ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:05.261033 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:05.262264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-94c06e77-dcc1-45d0-a6dd-4fa89911929f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:05.264174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 919/1863] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:04 2022] DELETE /identity/v3/projects/0948b03ac360454490482c4a7ef7c7ae => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:05.270263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe030f-ac0c-42d3-9afe-e18d9c6fac44 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:05.270716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe030f-ac0c-42d3-9afe-e18d9c6fac44 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:05.270953 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe030f-ac0c-42d3-9afe-e18d9c6fac44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:05.294572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-babe030f-ac0c-42d3-9afe-e18d9c6fac44 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:05.295814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-babe030f-ac0c-42d3-9afe-e18d9c6fac44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:05.337606 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802a8d8-65ae-4bf4-b97f-dfa54716a612 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:05.338010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802a8d8-65ae-4bf4-b97f-dfa54716a612 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:05.338301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802a8d8-65ae-4bf4-b97f-dfa54716a612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:05.388844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 945/1864] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:05.391936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e554229-d70e-4480-b4a7-603b46583388 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:05.392244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e554229-d70e-4480-b4a7-603b46583388 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:05.393646 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e554229-d70e-4480-b4a7-603b46583388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:05.395711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9802a8d8-65ae-4bf4-b97f-dfa54716a612 None None] MFA Rules not processed for user `eb2a7c1097e0421492e8460e9b64a27b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:05.401570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9802a8d8-65ae-4bf4-b97f-dfa54716a612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:05.430280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2e554229-d70e-4480-b4a7-603b46583388 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:05.438813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2e554229-d70e-4480-b4a7-603b46583388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:05.450191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 946/1865] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:05.454627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e554229-d70e-4480-b4a7-603b46583388 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:05.469967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e554229-d70e-4480-b4a7-603b46583388 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:05.476810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e920e-ddeb-4e95-974a-3fbe43afca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:05.477533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e920e-ddeb-4e95-974a-3fbe43afca40 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:05.477824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e920e-ddeb-4e95-974a-3fbe43afca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:05.478105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e920e-ddeb-4e95-974a-3fbe43afca40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:05.478825 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e920e-ddeb-4e95-974a-3fbe43afca40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:29:05.479123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e920e-ddeb-4e95-974a-3fbe43afca40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:05.480999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e920e-ddeb-4e95-974a-3fbe43afca40 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:05.521541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 920/1866] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:05.522162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 947/1867] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:29:05.535130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802a8d8-65ae-4bf4-b97f-dfa54716a612 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:05.547968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802a8d8-65ae-4bf4-b97f-dfa54716a612 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:05.553496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081e52f-5a00-4e0c-bee1-8d703e140c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:05.554136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081e52f-5a00-4e0c-bee1-8d703e140c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:05.554422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081e52f-5a00-4e0c-bee1-8d703e140c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:05.557080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081e52f-5a00-4e0c-bee1-8d703e140c93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:05.557080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081e52f-5a00-4e0c-bee1-8d703e140c93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:05.557563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081e52f-5a00-4e0c-bee1-8d703e140c93 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:05.647640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 921/1868] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:05 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:05.651125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e920e-ddeb-4e95-974a-3fbe43afca40 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:05.676256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e920e-ddeb-4e95-974a-3fbe43afca40 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:05.760314 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5ea52-775d-4bfb-9546-be70f07c6c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:05.761903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5ea52-775d-4bfb-9546-be70f07c6c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:05.762333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5ea52-775d-4bfb-9546-be70f07c6c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:05.762710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5ea52-775d-4bfb-9546-be70f07c6c5d admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:05.763499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5ea52-775d-4bfb-9546-be70f07c6c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:05.765753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5ea52-775d-4bfb-9546-be70f07c6c5d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:05.890756 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 948/1869] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:05 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:05.895078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081e52f-5a00-4e0c-bee1-8d703e140c93 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:05.906749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081e52f-5a00-4e0c-bee1-8d703e140c93 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:05.911415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5b18a-d051-4b12-bddb-72abd5aa39a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:05.912044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5b18a-d051-4b12-bddb-72abd5aa39a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:05.912319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5b18a-d051-4b12-bddb-72abd5aa39a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:05.912594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5b18a-d051-4b12-bddb-72abd5aa39a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:05.913368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5b18a-d051-4b12-bddb-72abd5aa39a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:05.915009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5b18a-d051-4b12-bddb-72abd5aa39a1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:05.929551 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 922/1870] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:29:05.930742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5ea52-775d-4bfb-9546-be70f07c6c5d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:05.941583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5ea52-775d-4bfb-9546-be70f07c6c5d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:05.946209 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b85ae-0eb4-4699-a600-8327221a35a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:05.946829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b85ae-0eb4-4699-a600-8327221a35a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:05.947109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b85ae-0eb4-4699-a600-8327221a35a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:05.947385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b85ae-0eb4-4699-a600-8327221a35a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:05.948081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b85ae-0eb4-4699-a600-8327221a35a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:05.949737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b85ae-0eb4-4699-a600-8327221a35a1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:05.961123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 949/1871] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:29:05.967657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5b18a-d051-4b12-bddb-72abd5aa39a1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:05.978275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5b18a-d051-4b12-bddb-72abd5aa39a1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:05.983871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ef603-7f1f-476b-9dbe-77358ff69ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:05.984585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ef603-7f1f-476b-9dbe-77358ff69ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:05.985460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ef603-7f1f-476b-9dbe-77358ff69ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:05.985864 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ef603-7f1f-476b-9dbe-77358ff69ca0 admin admin] PATH_INFO: `/v3/projects/263732b41c834d6c814e880a9711d135/users/688a2975aaf941329d96e4ee523fb827/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:06.003560 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ef603-7f1f-476b-9dbe-77358ff69ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=263732b41c834d6c814e880a9711d135, user_id=688a2975aaf941329d96e4ee523fb827, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:06.006585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ef603-7f1f-476b-9dbe-77358ff69ca0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:06.151233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 923/1872] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:05 2022] PUT /identity/v3/projects/263732b41c834d6c814e880a9711d135/users/688a2975aaf941329d96e4ee523fb827/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:06.164306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a6600-c3e1-427c-a8a0-6c41f4273662 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:06.165250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a6600-c3e1-427c-a8a0-6c41f4273662 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:06.170046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a6600-c3e1-427c-a8a0-6c41f4273662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:06.202944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-a78a6600-c3e1-427c-a8a0-6c41f4273662 None None] MFA Rules not processed for user `688a2975aaf941329d96e4ee523fb827`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:06.204867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a78a6600-c3e1-427c-a8a0-6c41f4273662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:06.278523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 950/1873] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:06.291203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ef603-7f1f-476b-9dbe-77358ff69ca0 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:06.293122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ab5ef603-7f1f-476b-9dbe-77358ff69ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:06.399558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ef603-7f1f-476b-9dbe-77358ff69ca0 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:06.405472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8208261-3c6a-454f-babc-108aee6c882d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:06.406130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8208261-3c6a-454f-babc-108aee6c882d service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:06.406419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8208261-3c6a-454f-babc-108aee6c882d service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:06.407665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8208261-3c6a-454f-babc-108aee6c882d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:06.417513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8208261-3c6a-454f-babc-108aee6c882d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:06.420318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8208261-3c6a-454f-babc-108aee6c882d service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:06.432715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 924/1874] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:07.182192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:07.628250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:09.089200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a6600-c3e1-427c-a8a0-6c41f4273662 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:09.091338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a78a6600-c3e1-427c-a8a0-6c41f4273662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:09.223236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a6600-c3e1-427c-a8a0-6c41f4273662 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:09.231410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:09.234301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:09.234301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:09.234301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] PATH_INFO: `/v3/users/2c64f0088817460394490a1bed2c258e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:09.234301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c64f0088817460394490a1bed2c258e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:09.236011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:09.561088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c64f0088817460394490a1bed2c258e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:09.574396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c64f0088817460394490a1bed2c258e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:09.575303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c64f0088817460394490a1bed2c258e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:09.582985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c64f0088817460394490a1bed2c258e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:09.615564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:09.673230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c64f0088817460394490a1bed2c258e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:09.681063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 951/1875] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:09 2022] DELETE /identity/v3/users/2c64f0088817460394490a1bed2c258e => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:09.687042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690aeb3-2685-4c3a-aac7-1ff5a435c4d8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:09.688459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690aeb3-2685-4c3a-aac7-1ff5a435c4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:09.688903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690aeb3-2685-4c3a-aac7-1ff5a435c4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:09.707967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f690aeb3-2685-4c3a-aac7-1ff5a435c4d8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:09.709183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f690aeb3-2685-4c3a-aac7-1ff5a435c4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:09.806949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 925/1876] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:09.823373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:09.826698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:09.903835 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f317d-51bb-4268-93e3-9e53adc37223 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:09.907631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbebc3d-e101-4f6c-82ab-261aaa1b057e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:09.908258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbebc3d-e101-4f6c-82ab-261aaa1b057e service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:09.908544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbebc3d-e101-4f6c-82ab-261aaa1b057e service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:09.908816 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbebc3d-e101-4f6c-82ab-261aaa1b057e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:09.918205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:09.918347 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbebc3d-e101-4f6c-82ab-261aaa1b057e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:09.919635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbebc3d-e101-4f6c-82ab-261aaa1b057e service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:09.929290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 952/1877] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:10.062421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690aeb3-2685-4c3a-aac7-1ff5a435c4d8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:10.064019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f690aeb3-2685-4c3a-aac7-1ff5a435c4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:10.076690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690aeb3-2685-4c3a-aac7-1ff5a435c4d8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:10.081297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:10.081967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:10.082264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:10.082548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] PATH_INFO: `/v3/users/688a2975aaf941329d96e4ee523fb827` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:10.083734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] RBAC: Authorizing `identity:delete_user(user_id=688a2975aaf941329d96e4ee523fb827)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:10.085717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:10.377344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '688a2975aaf941329d96e4ee523fb827'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:10.391416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '688a2975aaf941329d96e4ee523fb827'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:10.398779 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '688a2975aaf941329d96e4ee523fb827'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:10.502043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '688a2975aaf941329d96e4ee523fb827'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:10.502701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f9fd7112-c547-408b-ad2f-1c2dd39a716e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '688a2975aaf941329d96e4ee523fb827'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:10.505387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbebc3d-e101-4f6c-82ab-261aaa1b057e service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:10.508552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2cbebc3d-e101-4f6c-82ab-261aaa1b057e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:10.512584 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 926/1878] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:10 2022] DELETE /identity/v3/users/688a2975aaf941329d96e4ee523fb827 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:10.517571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c3f6-5168-4258-a319-fc9954fd6b95 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:10.518112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c3f6-5168-4258-a319-fc9954fd6b95 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:10.518706 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c3f6-5168-4258-a319-fc9954fd6b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:10.523565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbebc3d-e101-4f6c-82ab-261aaa1b057e service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:10.541253 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7740c3f6-5168-4258-a319-fc9954fd6b95 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:10.542677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7740c3f6-5168-4258-a319-fc9954fd6b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:10.605276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:10.606183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:10.606482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:10.606847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] PATH_INFO: `/v3/projects/7effa0ed8b924e2c9e305aec5c5bd79a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:10.608149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7effa0ed8b924e2c9e305aec5c5bd79a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:10.610184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:10.669140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 927/1879] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:10.726628 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740c3f6-5168-4258-a319-fc9954fd6b95 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:10.730377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7740c3f6-5168-4258-a319-fc9954fd6b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:10.742071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7effa0ed8b924e2c9e305aec5c5bd79a', 'request_id': 'req-ad83d88e-0da8-4235-9d8d-8bac813a9952'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:10.748060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:10.758984 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7effa0ed8b924e2c9e305aec5c5bd79a', 'request_id': 'req-ad83d88e-0da8-4235-9d8d-8bac813a9952'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:10.828865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7effa0ed8b924e2c9e305aec5c5bd79a', 'request_id': 'req-ad83d88e-0da8-4235-9d8d-8bac813a9952'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:10.855206 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] The token cache is being invalidate because project 7effa0ed8b924e2c9e305aec5c5bd79a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:10.857251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:10.858520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740c3f6-5168-4258-a319-fc9954fd6b95 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:10.858840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:10.865263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 953/1880] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:10 2022] DELETE /identity/v3/projects/7effa0ed8b924e2c9e305aec5c5bd79a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:10.869368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:10.874509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:10.963597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be8511-d25c-41eb-a8eb-402477b4c1d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:10.964651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be8511-d25c-41eb-a8eb-402477b4c1d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:10.965335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be8511-d25c-41eb-a8eb-402477b4c1d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:10.965853 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be8511-d25c-41eb-a8eb-402477b4c1d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:10.969620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a4be8511-d25c-41eb-a8eb-402477b4c1d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:11.008301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83d88e-0da8-4235-9d8d-8bac813a9952 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:11.014658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:11.014658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:11.015129 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:11.015129 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] PATH_INFO: `/v3/users/0ef65b9e1d9c457784c6f70054f74509` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:11.018171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ef65b9e1d9c457784c6f70054f74509)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:11.020418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:11.104291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be8511-d25c-41eb-a8eb-402477b4c1d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:11.106198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be8511-d25c-41eb-a8eb-402477b4c1d5 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:11.124890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 928/1881] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:11.237627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ef65b9e1d9c457784c6f70054f74509'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:11.286386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ef65b9e1d9c457784c6f70054f74509'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:11.287596 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ef65b9e1d9c457784c6f70054f74509'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:11.296711 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ef65b9e1d9c457784c6f70054f74509'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:11.358428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ef65b9e1d9c457784c6f70054f74509'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:11.375332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 954/1882] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:10 2022] DELETE /identity/v3/users/0ef65b9e1d9c457784c6f70054f74509 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:11.531058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:12.155925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be8511-d25c-41eb-a8eb-402477b4c1d5 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:12.175543 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be8511-d25c-41eb-a8eb-402477b4c1d5 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:12.193109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:12.195231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:12.205750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:12.287367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:12.288779 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.289278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.289832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] PATH_INFO: `/v3/projects/14adc84a39404a9d8776354905729b28` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.291401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14adc84a39404a9d8776354905729b28)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:12.293636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:12.349605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873c71d-3bd5-4ce8-a5f6-4596ab8c1ce2 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:12.358458 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:12.360462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.361923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.361923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] PATH_INFO: `/v3/projects/263732b41c834d6c814e880a9711d135` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.363054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=263732b41c834d6c814e880a9711d135)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:12.365361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:12.473749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14adc84a39404a9d8776354905729b28', 'request_id': 'req-bf17f3df-0504-458d-840a-4de1baedbcc8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:12.489587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14adc84a39404a9d8776354905729b28', 'request_id': 'req-bf17f3df-0504-458d-840a-4de1baedbcc8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:12.504663 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14adc84a39404a9d8776354905729b28', 'request_id': 'req-bf17f3df-0504-458d-840a-4de1baedbcc8'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:12.532080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '263732b41c834d6c814e880a9711d135', 'request_id': 'req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:12.548961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '263732b41c834d6c814e880a9711d135', 'request_id': 'req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:12.631996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] The token cache is being invalidate because project 14adc84a39404a9d8776354905729b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:12.632319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:12.634053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf17f3df-0504-458d-840a-4de1baedbcc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:12.644548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 929/1883] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:12 2022] DELETE /identity/v3/projects/14adc84a39404a9d8776354905729b28 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:12.650353 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2422824-6fb3-47b0-a3b8-40dc38412a32 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.650779 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2422824-6fb3-47b0-a3b8-40dc38412a32 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.651075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2422824-6fb3-47b0-a3b8-40dc38412a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.673279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d2422824-6fb3-47b0-a3b8-40dc38412a32 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:12.675100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d2422824-6fb3-47b0-a3b8-40dc38412a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:12.731004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '263732b41c834d6c814e880a9711d135', 'request_id': 'req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:12.761350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] The token cache is being invalidate because project 263732b41c834d6c814e880a9711d135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:12.763294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:12.764123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cc7a5a8-3371-4aaf-93f4-bf473431fa1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:12.767575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 955/1884] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:12 2022] DELETE /identity/v3/projects/263732b41c834d6c814e880a9711d135 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:12.776901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c144-69a3-446c-ae2d-25e558e6323e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.777659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c144-69a3-446c-ae2d-25e558e6323e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.778171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c144-69a3-446c-ae2d-25e558e6323e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.778430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 930/1885] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:12.800742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083178a-c5a9-4dc3-92b7-66f0d5678f8f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.801067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083178a-c5a9-4dc3-92b7-66f0d5678f8f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.801412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083178a-c5a9-4dc3-92b7-66f0d5678f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.805279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-dcb2c144-69a3-446c-ae2d-25e558e6323e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:12.807882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-dcb2c144-69a3-446c-ae2d-25e558e6323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:12.818759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 956/1886] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:12.833167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ac19-598d-4641-ac9b-e95063538bf7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.833545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ac19-598d-4641-ac9b-e95063538bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.833770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ac19-598d-4641-ac9b-e95063538bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.843382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2083178a-c5a9-4dc3-92b7-66f0d5678f8f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:12.848991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2083178a-c5a9-4dc3-92b7-66f0d5678f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:12.872130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 931/1887] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:12.873533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-fbc9ac19-598d-4641-ac9b-e95063538bf7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:12.874785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-fbc9ac19-598d-4641-ac9b-e95063538bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:12.881350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083178a-c5a9-4dc3-92b7-66f0d5678f8f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:12.894965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083178a-c5a9-4dc3-92b7-66f0d5678f8f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:12.905074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac14489-0864-4b20-95df-66519923a491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:12.905759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 957/1888] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:12.906148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac14489-0864-4b20-95df-66519923a491 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.907126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac14489-0864-4b20-95df-66519923a491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.907455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac14489-0864-4b20-95df-66519923a491 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.908259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac14489-0864-4b20-95df-66519923a491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:29:12.908582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac14489-0864-4b20-95df-66519923a491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:12.910632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ac19-598d-4641-ac9b-e95063538bf7 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:12.916979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac14489-0864-4b20-95df-66519923a491 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:12.931053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ac19-598d-4641-ac9b-e95063538bf7 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:12.944436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429430f8-e232-4c36-b3f0-fc67d7af562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:12.945180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429430f8-e232-4c36-b3f0-fc67d7af562a admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.945714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429430f8-e232-4c36-b3f0-fc67d7af562a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.946002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429430f8-e232-4c36-b3f0-fc67d7af562a admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.947239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429430f8-e232-4c36-b3f0-fc67d7af562a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:29:12.947555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429430f8-e232-4c36-b3f0-fc67d7af562a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:12.949802 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429430f8-e232-4c36-b3f0-fc67d7af562a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:12.956453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 932/1889] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:29:12.958074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac14489-0864-4b20-95df-66519923a491 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:12.975598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac14489-0864-4b20-95df-66519923a491 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:12.988052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 958/1890] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:29:12.991310 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e500cba-bdf0-4ee2-b4eb-62d2725c3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:12.991610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429430f8-e232-4c36-b3f0-fc67d7af562a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:12.992230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e500cba-bdf0-4ee2-b4eb-62d2725c3da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.992668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e500cba-bdf0-4ee2-b4eb-62d2725c3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.993049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e500cba-bdf0-4ee2-b4eb-62d2725c3da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.993887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e500cba-bdf0-4ee2-b4eb-62d2725c3da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:12.996391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e500cba-bdf0-4ee2-b4eb-62d2725c3da1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.020537 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429430f8-e232-4c36-b3f0-fc67d7af562a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:13.030165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf6354-792f-462b-8cd4-8a7138dd8972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:13.032326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf6354-792f-462b-8cd4-8a7138dd8972 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.032696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf6354-792f-462b-8cd4-8a7138dd8972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.032988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf6354-792f-462b-8cd4-8a7138dd8972 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.033785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf6354-792f-462b-8cd4-8a7138dd8972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:13.036656 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf6354-792f-462b-8cd4-8a7138dd8972 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.089166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:13.135487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 933/1891] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:12 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:13.139884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e500cba-bdf0-4ee2-b4eb-62d2725c3da1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:13.214716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 959/1892] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:12 2022] POST /identity/v3/projects => generated 363 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:13.217742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf6354-792f-462b-8cd4-8a7138dd8972 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:13.306525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e500cba-bdf0-4ee2-b4eb-62d2725c3da1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:13.384222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf6354-792f-462b-8cd4-8a7138dd8972 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:13.402192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e79a7e-fa54-4597-a29d-75029d4401e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:13.403124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e79a7e-fa54-4597-a29d-75029d4401e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.403424 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e79a7e-fa54-4597-a29d-75029d4401e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.403712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e79a7e-fa54-4597-a29d-75029d4401e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.404655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e79a7e-fa54-4597-a29d-75029d4401e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:13.406462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e79a7e-fa54-4597-a29d-75029d4401e4 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.445423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69769214-af3a-4163-8179-483fa7314309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:13.446324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69769214-af3a-4163-8179-483fa7314309 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.446612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69769214-af3a-4163-8179-483fa7314309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.446893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69769214-af3a-4163-8179-483fa7314309 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.447716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69769214-af3a-4163-8179-483fa7314309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:13.450038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69769214-af3a-4163-8179-483fa7314309 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.511383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 960/1893] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:13 2022] POST /identity/v3/users => generated 282 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:13.514348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e79a7e-fa54-4597-a29d-75029d4401e4 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:13.528193 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e79a7e-fa54-4597-a29d-75029d4401e4 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:13.536964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a8294-f6fa-434b-9bd1-22b144311732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:13.536964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a8294-f6fa-434b-9bd1-22b144311732 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.537262 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 934/1894] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:13 2022] POST /identity/v3/users => generated 294 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:13.537384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a8294-f6fa-434b-9bd1-22b144311732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.537463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a8294-f6fa-434b-9bd1-22b144311732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.538290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a8294-f6fa-434b-9bd1-22b144311732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:13.540568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69769214-af3a-4163-8179-483fa7314309 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:13.540715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a8294-f6fa-434b-9bd1-22b144311732 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.555376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69769214-af3a-4163-8179-483fa7314309 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:13.566434 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364b3a-bf5a-481d-9fc4-d720d23c101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:13.567137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364b3a-bf5a-481d-9fc4-d720d23c101b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.567427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364b3a-bf5a-481d-9fc4-d720d23c101b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.567753 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364b3a-bf5a-481d-9fc4-d720d23c101b admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.568482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89364b3a-bf5a-481d-9fc4-d720d23c101b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:13.570254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89364b3a-bf5a-481d-9fc4-d720d23c101b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.581353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 961/1895] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:13.584922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a8294-f6fa-434b-9bd1-22b144311732 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:13.600958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a8294-f6fa-434b-9bd1-22b144311732 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:13.601230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 935/1896] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:29:13.606710 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364b3a-bf5a-481d-9fc4-d720d23c101b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:13.608001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26765ebf-c38c-433f-adc7-69d115561c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:13.610356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26765ebf-c38c-433f-adc7-69d115561c38 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.611025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26765ebf-c38c-433f-adc7-69d115561c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.617666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26765ebf-c38c-433f-adc7-69d115561c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.617666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26765ebf-c38c-433f-adc7-69d115561c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:13.617666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26765ebf-c38c-433f-adc7-69d115561c38 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.631478 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364b3a-bf5a-481d-9fc4-d720d23c101b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:13.642099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d2b20-b691-496c-8f47-cf9f22b2a42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:13.642458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d2b20-b691-496c-8f47-cf9f22b2a42b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.642744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d2b20-b691-496c-8f47-cf9f22b2a42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.643026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d2b20-b691-496c-8f47-cf9f22b2a42b admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.643750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d2b20-b691-496c-8f47-cf9f22b2a42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:13.645402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d2b20-b691-496c-8f47-cf9f22b2a42b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.655333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 962/1897] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:29:13.660629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26765ebf-c38c-433f-adc7-69d115561c38 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:13.661852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 936/1898] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:13.666600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d2b20-b691-496c-8f47-cf9f22b2a42b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:13.673103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:13.680416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26765ebf-c38c-433f-adc7-69d115561c38 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:13.683244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55d24b-f065-4b3a-82ca-2a745b32b717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:13.684020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55d24b-f065-4b3a-82ca-2a745b32b717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.684383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55d24b-f065-4b3a-82ca-2a745b32b717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.684735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55d24b-f065-4b3a-82ca-2a745b32b717 admin admin] PATH_INFO: `/v3/projects/310e955c99624dc88a38254e9a36ca20/users/189a5115ddf94081ad010a1a01df10ed/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.689294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d2b20-b691-496c-8f47-cf9f22b2a42b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:13.702511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da5641-3f44-4082-a288-d264e2e9c0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:13.704110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da5641-3f44-4082-a288-d264e2e9c0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.704565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da5641-3f44-4082-a288-d264e2e9c0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.709001 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da5641-3f44-4082-a288-d264e2e9c0c1 admin admin] PATH_INFO: `/v3/projects/9ddc89f69fae4de3979e73c78e222257/users/463caa15f5d24a9a833beadc09242227/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.709139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55d24b-f065-4b3a-82ca-2a745b32b717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=310e955c99624dc88a38254e9a36ca20, user_id=189a5115ddf94081ad010a1a01df10ed, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:13.711401 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55d24b-f065-4b3a-82ca-2a745b32b717 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.725160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da5641-3f44-4082-a288-d264e2e9c0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ddc89f69fae4de3979e73c78e222257, user_id=463caa15f5d24a9a833beadc09242227, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:13.727667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da5641-3f44-4082-a288-d264e2e9c0c1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.821436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 963/1899] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:13 2022] PUT /identity/v3/projects/310e955c99624dc88a38254e9a36ca20/users/189a5115ddf94081ad010a1a01df10ed/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:13.831853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3ca29-ea6e-4dfa-9e96-1f1a24d08851 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.832139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3ca29-ea6e-4dfa-9e96-1f1a24d08851 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.832428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3ca29-ea6e-4dfa-9e96-1f1a24d08851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.860475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 937/1900] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:13 2022] PUT /identity/v3/projects/9ddc89f69fae4de3979e73c78e222257/users/463caa15f5d24a9a833beadc09242227/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:13.865491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-97a3ca29-ea6e-4dfa-9e96-1f1a24d08851 None None] MFA Rules not processed for user `189a5115ddf94081ad010a1a01df10ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:13.869442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-97a3ca29-ea6e-4dfa-9e96-1f1a24d08851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:13.879459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c942-1882-478a-b2c8-481cc7e8b62e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:13.879845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c942-1882-478a-b2c8-481cc7e8b62e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:13.880203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c942-1882-478a-b2c8-481cc7e8b62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:13.924394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0383c942-1882-478a-b2c8-481cc7e8b62e None None] MFA Rules not processed for user `463caa15f5d24a9a833beadc09242227`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:13.925827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0383c942-1882-478a-b2c8-481cc7e8b62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:13.970229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:13.971105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 964/1901] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:13 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:13.985324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3ca29-ea6e-4dfa-9e96-1f1a24d08851 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:13.998483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-97a3ca29-ea6e-4dfa-9e96-1f1a24d08851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:14.061684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 938/1902] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:14.066770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383c942-1882-478a-b2c8-481cc7e8b62e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:14.143658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3ca29-ea6e-4dfa-9e96-1f1a24d08851 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:14.153082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71663a-a188-4b15-820b-8035c52317a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:14.154123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71663a-a188-4b15-820b-8035c52317a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:14.158981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71663a-a188-4b15-820b-8035c52317a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:14.158981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71663a-a188-4b15-820b-8035c52317a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:14.167360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71663a-a188-4b15-820b-8035c52317a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:14.169198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71663a-a188-4b15-820b-8035c52317a5 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:14.169501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383c942-1882-478a-b2c8-481cc7e8b62e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:14.177950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b032ee4-b551-4567-a393-c5d45b8f6b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:14.178596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b032ee4-b551-4567-a393-c5d45b8f6b37 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:14.178879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b032ee4-b551-4567-a393-c5d45b8f6b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:14.179157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b032ee4-b551-4567-a393-c5d45b8f6b37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:14.179840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b032ee4-b551-4567-a393-c5d45b8f6b37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:14.181834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b032ee4-b551-4567-a393-c5d45b8f6b37 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:14.194220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 965/1903] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:14.314528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 939/1904] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:14 2022] POST /identity/v3/projects => generated 387 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:14.318732 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71663a-a188-4b15-820b-8035c52317a5 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:14.330540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71663a-a188-4b15-820b-8035c52317a5 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:14.408142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c307ba-1bce-4e73-9d6b-4fb2ecce06e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:14.410700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c307ba-1bce-4e73-9d6b-4fb2ecce06e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:14.411195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c307ba-1bce-4e73-9d6b-4fb2ecce06e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:14.411479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c307ba-1bce-4e73-9d6b-4fb2ecce06e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:14.412817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c307ba-1bce-4e73-9d6b-4fb2ecce06e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:14.415103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c307ba-1bce-4e73-9d6b-4fb2ecce06e3 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:14.499483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 966/1905] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:14 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:14.501377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b032ee4-b551-4567-a393-c5d45b8f6b37 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:14.518076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b032ee4-b551-4567-a393-c5d45b8f6b37 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:14.523064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c20b0d-c93e-4346-bac9-c7b5d5d0e0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:14.523761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c20b0d-c93e-4346-bac9-c7b5d5d0e0fc admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:14.524051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c20b0d-c93e-4346-bac9-c7b5d5d0e0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:14.524348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c20b0d-c93e-4346-bac9-c7b5d5d0e0fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:14.525055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c20b0d-c93e-4346-bac9-c7b5d5d0e0fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:14.526731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c20b0d-c93e-4346-bac9-c7b5d5d0e0fc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:14.538013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 940/1906] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:14.542557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c307ba-1bce-4e73-9d6b-4fb2ecce06e3 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:14.554109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c307ba-1bce-4e73-9d6b-4fb2ecce06e3 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:14.559451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c42a8-586c-4cdc-b512-11050c7945d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:14.560217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c42a8-586c-4cdc-b512-11050c7945d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:14.560513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c42a8-586c-4cdc-b512-11050c7945d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:14.560797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c42a8-586c-4cdc-b512-11050c7945d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:14.561596 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c42a8-586c-4cdc-b512-11050c7945d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:14.563773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c42a8-586c-4cdc-b512-11050c7945d9 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:14.574703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 967/1907] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:14.581410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c20b0d-c93e-4346-bac9-c7b5d5d0e0fc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:14.592548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c20b0d-c93e-4346-bac9-c7b5d5d0e0fc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:14.602161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d5abb-2230-46a6-a680-01452b0ee99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:14.602161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d5abb-2230-46a6-a680-01452b0ee99d admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:14.602161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d5abb-2230-46a6-a680-01452b0ee99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:14.602161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d5abb-2230-46a6-a680-01452b0ee99d admin admin] PATH_INFO: `/v3/projects/8606627b74f44945aad4c8f3f167e6f6/users/e0b1991aa99a4a3f8cb6b049c51b62de/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:14.619547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d5abb-2230-46a6-a680-01452b0ee99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8606627b74f44945aad4c8f3f167e6f6, user_id=e0b1991aa99a4a3f8cb6b049c51b62de, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:14.622408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d5abb-2230-46a6-a680-01452b0ee99d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:14.708405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 941/1908] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:14 2022] PUT /identity/v3/projects/8606627b74f44945aad4c8f3f167e6f6/users/e0b1991aa99a4a3f8cb6b049c51b62de/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:14.719383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967816b-b090-4e4a-be82-fd681341507b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:14.721981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967816b-b090-4e4a-be82-fd681341507b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:14.721981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967816b-b090-4e4a-be82-fd681341507b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:14.751515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e967816b-b090-4e4a-be82-fd681341507b None None] MFA Rules not processed for user `e0b1991aa99a4a3f8cb6b049c51b62de`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:14.753251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e967816b-b090-4e4a-be82-fd681341507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:14.846328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 968/1909] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:14.852240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d5abb-2230-46a6-a680-01452b0ee99d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:14.952840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:14.971069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d5abb-2230-46a6-a680-01452b0ee99d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:14.976155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48d7db-f9f7-41fb-a618-116645e24260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:14.976877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48d7db-f9f7-41fb-a618-116645e24260 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:14.977070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48d7db-f9f7-41fb-a618-116645e24260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:14.977392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48d7db-f9f7-41fb-a618-116645e24260 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:14.978198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48d7db-f9f7-41fb-a618-116645e24260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:14.989561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48d7db-f9f7-41fb-a618-116645e24260 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:15.105349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 942/1910] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:14 2022] POST /identity/v3/projects => generated 385 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:15.108879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967816b-b090-4e4a-be82-fd681341507b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:15.123923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967816b-b090-4e4a-be82-fd681341507b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:15.198091 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:15.208914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b266ed5-479b-4d5f-9c23-b823e432e4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:15.209942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266ed5-479b-4d5f-9c23-b823e432e4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:15.209942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266ed5-479b-4d5f-9c23-b823e432e4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:15.210225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266ed5-479b-4d5f-9c23-b823e432e4cb admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:15.210992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b266ed5-479b-4d5f-9c23-b823e432e4cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:15.213032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b266ed5-479b-4d5f-9c23-b823e432e4cb admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:15.289382 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 969/1911] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:15 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:15.290757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48d7db-f9f7-41fb-a618-116645e24260 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:15.304374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48d7db-f9f7-41fb-a618-116645e24260 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:15.311643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408f81e-90ca-4967-999d-fee5ce1c1829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:15.313335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408f81e-90ca-4967-999d-fee5ce1c1829 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:15.313633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408f81e-90ca-4967-999d-fee5ce1c1829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:15.313920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408f81e-90ca-4967-999d-fee5ce1c1829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:15.314655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408f81e-90ca-4967-999d-fee5ce1c1829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:15.325898 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408f81e-90ca-4967-999d-fee5ce1c1829 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:15.332242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 943/1912] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:15.336586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b266ed5-479b-4d5f-9c23-b823e432e4cb admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:15.354774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b266ed5-479b-4d5f-9c23-b823e432e4cb admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:15.364135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fdce9-5cfe-4a97-a7bd-20b10a1aa4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:15.365507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fdce9-5cfe-4a97-a7bd-20b10a1aa4fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:15.366119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fdce9-5cfe-4a97-a7bd-20b10a1aa4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:15.366625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fdce9-5cfe-4a97-a7bd-20b10a1aa4fc admin admin] PATH_INFO: `/v3/projects/f954555afb2e47e7b546db5b8608db7c/users/d12b7e808fc04c0e867d22b49b8f5312/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:15.384389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055fdce9-5cfe-4a97-a7bd-20b10a1aa4fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f954555afb2e47e7b546db5b8608db7c, user_id=d12b7e808fc04c0e867d22b49b8f5312, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:15.387177 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055fdce9-5cfe-4a97-a7bd-20b10a1aa4fc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:15.473015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 970/1913] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:15 2022] PUT /identity/v3/projects/f954555afb2e47e7b546db5b8608db7c/users/d12b7e808fc04c0e867d22b49b8f5312/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:15.476922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408f81e-90ca-4967-999d-fee5ce1c1829 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:15.488830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408f81e-90ca-4967-999d-fee5ce1c1829 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:15.514043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:15.558690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b26d3-81cb-497e-894b-d7d352df5f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:15.559513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b26d3-81cb-497e-894b-d7d352df5f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:15.559655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b26d3-81cb-497e-894b-d7d352df5f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:15.559880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b26d3-81cb-497e-894b-d7d352df5f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:15.560925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b26d3-81cb-497e-894b-d7d352df5f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:15.562452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b26d3-81cb-497e-894b-d7d352df5f23 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:15.573428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 944/1914] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:15.577471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fdce9-5cfe-4a97-a7bd-20b10a1aa4fc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:15.589632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fdce9-5cfe-4a97-a7bd-20b10a1aa4fc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:15.595565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd37aa-c6d0-4b12-86e1-b1626490a9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:15.596228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd37aa-c6d0-4b12-86e1-b1626490a9de admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:15.596518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd37aa-c6d0-4b12-86e1-b1626490a9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:15.596854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd37aa-c6d0-4b12-86e1-b1626490a9de admin admin] PATH_INFO: `/v3/projects/f954555afb2e47e7b546db5b8608db7c/users/d12b7e808fc04c0e867d22b49b8f5312/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:15.599163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd37aa-c6d0-4b12-86e1-b1626490a9de admin admin] RBAC: Authorizing `identity:create_grant(project_id=f954555afb2e47e7b546db5b8608db7c, user_id=d12b7e808fc04c0e867d22b49b8f5312, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:15.602215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd37aa-c6d0-4b12-86e1-b1626490a9de admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:15.623232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 971/1915] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:15 2022] PUT /identity/v3/projects/f954555afb2e47e7b546db5b8608db7c/users/d12b7e808fc04c0e867d22b49b8f5312/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:15.625433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550742de-d2b3-4d30-af62-201fd35ab4c9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:15.625735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550742de-d2b3-4d30-af62-201fd35ab4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:15.626022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550742de-d2b3-4d30-af62-201fd35ab4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:15.665168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-550742de-d2b3-4d30-af62-201fd35ab4c9 None None] MFA Rules not processed for user `d12b7e808fc04c0e867d22b49b8f5312`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:15.665168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-550742de-d2b3-4d30-af62-201fd35ab4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:15.701087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:15.740505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 945/1916] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:15.753266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd37aa-c6d0-4b12-86e1-b1626490a9de admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:15.813140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd37aa-c6d0-4b12-86e1-b1626490a9de admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:15.817179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dfefd-e828-4bdd-9a8a-682e4b1d58e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:15.817858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dfefd-e828-4bdd-9a8a-682e4b1d58e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:15.818146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dfefd-e828-4bdd-9a8a-682e4b1d58e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:15.818420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dfefd-e828-4bdd-9a8a-682e4b1d58e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:15.832518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dfefd-e828-4bdd-9a8a-682e4b1d58e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:15.832518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dfefd-e828-4bdd-9a8a-682e4b1d58e4 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:15.840273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 972/1917] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:17.189935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550742de-d2b3-4d30-af62-201fd35ab4c9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:17.200120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550742de-d2b3-4d30-af62-201fd35ab4c9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:17.205623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b2c5f-f080-4a98-b238-5b197643315c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:17.206463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b2c5f-f080-4a98-b238-5b197643315c service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:17.206901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b2c5f-f080-4a98-b238-5b197643315c service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:17.207331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b2c5f-f080-4a98-b238-5b197643315c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:17.305855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b2c5f-f080-4a98-b238-5b197643315c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:17.307380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b2c5f-f080-4a98-b238-5b197643315c service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:17.338800 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 946/1918] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:22.641943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dfefd-e828-4bdd-9a8a-682e4b1d58e4 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:22.651044 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dfefd-e828-4bdd-9a8a-682e4b1d58e4 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:22.657434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedd1f8-aff3-4030-af80-6a4d0599cfab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:22.657434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedd1f8-aff3-4030-af80-6a4d0599cfab service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:22.657434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedd1f8-aff3-4030-af80-6a4d0599cfab service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:22.657434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedd1f8-aff3-4030-af80-6a4d0599cfab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:22.748291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedd1f8-aff3-4030-af80-6a4d0599cfab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:22.748291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedd1f8-aff3-4030-af80-6a4d0599cfab service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:22.757739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 973/1919] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:27.315047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=41 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:28.292716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84f07d-709c-4010-be3a-fd3f639e00e9 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:28.292716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84f07d-709c-4010-be3a-fd3f639e00e9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:28.293238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84f07d-709c-4010-be3a-fd3f639e00e9 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:28.295160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 947/1920] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:28.323322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221bb49-cb46-4fe1-895f-c869825f920b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:28.323928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221bb49-cb46-4fe1-895f-c869825f920b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:28.323928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221bb49-cb46-4fe1-895f-c869825f920b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:28.379535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b221bb49-cb46-4fe1-895f-c869825f920b None None] MFA Rules not processed for user `244a8787df7641f2bac9b51213ddb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:28.380757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b221bb49-cb46-4fe1-895f-c869825f920b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:28.458959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 974/1921] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:29:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:28.477378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84f07d-709c-4010-be3a-fd3f639e00e9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:28.479213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6f84f07d-709c-4010-be3a-fd3f639e00e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:28.481622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221bb49-cb46-4fe1-895f-c869825f920b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:28.484311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b221bb49-cb46-4fe1-895f-c869825f920b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:28.513031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221bb49-cb46-4fe1-895f-c869825f920b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:28.521137 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204074c8-2b38-440c-848d-8f14c04bb117 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:28.521917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204074c8-2b38-440c-848d-8f14c04bb117 service placement] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:28.522283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204074c8-2b38-440c-848d-8f14c04bb117 service placement] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:28.522666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204074c8-2b38-440c-848d-8f14c04bb117 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:28.533407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204074c8-2b38-440c-848d-8f14c04bb117 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:28.535152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204074c8-2b38-440c-848d-8f14c04bb117 service placement] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:28.545927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 975/1922] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:29:28.574788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84f07d-709c-4010-be3a-fd3f639e00e9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:28.580325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:28.580969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:28.585049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:28.585049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] PATH_INFO: `/v3/users/eda0cfe646954a9a8476b346ac9fed72` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:28.585049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda0cfe646954a9a8476b346ac9fed72)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:28.585712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:28.912727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda0cfe646954a9a8476b346ac9fed72'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:28.927394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda0cfe646954a9a8476b346ac9fed72'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:28.934798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda0cfe646954a9a8476b346ac9fed72'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:29.019877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda0cfe646954a9a8476b346ac9fed72'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:29.020453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db018d63-7274-4fc6-982e-54c1b7c4a766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda0cfe646954a9a8476b346ac9fed72'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:29.027039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 948/1923] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:28 2022] DELETE /identity/v3/users/eda0cfe646954a9a8476b346ac9fed72 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:29.344117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204074c8-2b38-440c-848d-8f14c04bb117 service placement] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:29.372592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204074c8-2b38-440c-848d-8f14c04bb117 service placement] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:29.392553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:29.460322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:29.461077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:29.461480 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:29.461934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] PATH_INFO: `/v3/projects/58565c8c6b5743278fb155d39ed65e3d` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:29.464423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] RBAC: Authorizing `identity:delete_project(project_id=58565c8c6b5743278fb155d39ed65e3d)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:29.466912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:29.615256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58565c8c6b5743278fb155d39ed65e3d', 'request_id': 'req-5befd6b9-a33e-4496-abff-789fbd9cd12d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:29.621675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58565c8c6b5743278fb155d39ed65e3d', 'request_id': 'req-5befd6b9-a33e-4496-abff-789fbd9cd12d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:29.677087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58565c8c6b5743278fb155d39ed65e3d', 'request_id': 'req-5befd6b9-a33e-4496-abff-789fbd9cd12d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:29.698839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] The token cache is being invalidate because project 58565c8c6b5743278fb155d39ed65e3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:29.699514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:29.700290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-5befd6b9-a33e-4496-abff-789fbd9cd12d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:29.705343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 976/1924] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:29 2022] DELETE /identity/v3/projects/58565c8c6b5743278fb155d39ed65e3d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:29.707912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3a3d1-a24c-4d05-b023-09133ba012f7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:29.708313 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3a3d1-a24c-4d05-b023-09133ba012f7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:29.708514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3a3d1-a24c-4d05-b023-09133ba012f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:29.730481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b6e3a3d1-a24c-4d05-b023-09133ba012f7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:29.731881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b6e3a3d1-a24c-4d05-b023-09133ba012f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:29.798179 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 949/1925] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:29.806013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aa429-e53e-4aed-9cb6-e31e485654ef None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:29.806306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aa429-e53e-4aed-9cb6-e31e485654ef None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:29.806639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aa429-e53e-4aed-9cb6-e31e485654ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:29.827372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-760aa429-e53e-4aed-9cb6-e31e485654ef None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:29.829112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-760aa429-e53e-4aed-9cb6-e31e485654ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:29.837219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 977/1926] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:29.841857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3a3d1-a24c-4d05-b023-09133ba012f7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:29.852917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3a3d1-a24c-4d05-b023-09133ba012f7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:29.857843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb444e-4eaf-4200-a88f-ff2c33ecd686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:29.858549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb444e-4eaf-4200-a88f-ff2c33ecd686 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:29.859367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb444e-4eaf-4200-a88f-ff2c33ecd686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:29.859720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb444e-4eaf-4200-a88f-ff2c33ecd686 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:29.860561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb444e-4eaf-4200-a88f-ff2c33ecd686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:29:29.860912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb444e-4eaf-4200-a88f-ff2c33ecd686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:29.862535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb444e-4eaf-4200-a88f-ff2c33ecd686 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:29.875564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 950/1927] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:29:29.880214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760aa429-e53e-4aed-9cb6-e31e485654ef None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:29.890405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760aa429-e53e-4aed-9cb6-e31e485654ef None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:29.895256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a28867-2326-44b5-8d9d-f20b39f57454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:29.895870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a28867-2326-44b5-8d9d-f20b39f57454 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:29.896157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a28867-2326-44b5-8d9d-f20b39f57454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:29.896432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a28867-2326-44b5-8d9d-f20b39f57454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:29.897112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a28867-2326-44b5-8d9d-f20b39f57454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:29.899058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a28867-2326-44b5-8d9d-f20b39f57454 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:30.040352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 978/1928] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:29 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:30.044538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb444e-4eaf-4200-a88f-ff2c33ecd686 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:30.056314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb444e-4eaf-4200-a88f-ff2c33ecd686 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:30.061530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:30.119212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4e130-9e41-4b81-96ce-84cc9d2d19fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:30.119852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4e130-9e41-4b81-96ce-84cc9d2d19fb admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.120144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4e130-9e41-4b81-96ce-84cc9d2d19fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.120427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4e130-9e41-4b81-96ce-84cc9d2d19fb admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.121174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4e130-9e41-4b81-96ce-84cc9d2d19fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:30.123352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4e130-9e41-4b81-96ce-84cc9d2d19fb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:30.223005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 951/1929] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:30 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:30.227464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a28867-2326-44b5-8d9d-f20b39f57454 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:30.241441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a28867-2326-44b5-8d9d-f20b39f57454 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:30.246789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a27b5a-d0dc-4e2d-8f25-785644cf078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:30.247296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a27b5a-d0dc-4e2d-8f25-785644cf078c admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.254013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a27b5a-d0dc-4e2d-8f25-785644cf078c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.254013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a27b5a-d0dc-4e2d-8f25-785644cf078c admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.254013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a27b5a-d0dc-4e2d-8f25-785644cf078c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:30.254013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a27b5a-d0dc-4e2d-8f25-785644cf078c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:30.265251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 979/1930] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:30.268538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4e130-9e41-4b81-96ce-84cc9d2d19fb admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:30.279983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4e130-9e41-4b81-96ce-84cc9d2d19fb admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:30.285367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884b510-2f46-4fa0-a0f2-4723a9f35e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:30.286012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884b510-2f46-4fa0-a0f2-4723a9f35e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.286398 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884b510-2f46-4fa0-a0f2-4723a9f35e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.286798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884b510-2f46-4fa0-a0f2-4723a9f35e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.287555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884b510-2f46-4fa0-a0f2-4723a9f35e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:30.289294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884b510-2f46-4fa0-a0f2-4723a9f35e56 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:30.306382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 952/1931] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:30.306486 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a27b5a-d0dc-4e2d-8f25-785644cf078c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:30.307299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884b510-2f46-4fa0-a0f2-4723a9f35e56 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:30.309003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f884b510-2f46-4fa0-a0f2-4723a9f35e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:30.316952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a27b5a-d0dc-4e2d-8f25-785644cf078c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:30.321837 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419d3d2-79ee-4290-b2b9-d63b8b319ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:30.322522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419d3d2-79ee-4290-b2b9-d63b8b319ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.322633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884b510-2f46-4fa0-a0f2-4723a9f35e56 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:30.322813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419d3d2-79ee-4290-b2b9-d63b8b319ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.323471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419d3d2-79ee-4290-b2b9-d63b8b319ff0 admin admin] PATH_INFO: `/v3/projects/99b364d50dbb463886c0430482d3b67c/users/64771ac5aa624027bd3bdc7c32bf8228/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.328428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:30.329072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.329388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.330975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] PATH_INFO: `/v3/users/463caa15f5d24a9a833beadc09242227` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.330975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=463caa15f5d24a9a833beadc09242227)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:30.332964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:30.340340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419d3d2-79ee-4290-b2b9-d63b8b319ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b364d50dbb463886c0430482d3b67c, user_id=64771ac5aa624027bd3bdc7c32bf8228, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:30.344717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419d3d2-79ee-4290-b2b9-d63b8b319ff0 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:30.433037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 980/1932] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:30 2022] PUT /identity/v3/projects/99b364d50dbb463886c0430482d3b67c/users/64771ac5aa624027bd3bdc7c32bf8228/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:30.443431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc62093-dcde-40c0-ba0e-f10d1c201e72 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.443728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc62093-dcde-40c0-ba0e-f10d1c201e72 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.444012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc62093-dcde-40c0-ba0e-f10d1c201e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.473317 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-8dc62093-dcde-40c0-ba0e-f10d1c201e72 None None] MFA Rules not processed for user `64771ac5aa624027bd3bdc7c32bf8228`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:30.474565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8dc62093-dcde-40c0-ba0e-f10d1c201e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:30.495328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:30.543926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 981/1933] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:30.549003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc62093-dcde-40c0-ba0e-f10d1c201e72 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:30.552176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '463caa15f5d24a9a833beadc09242227'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:30.565490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '463caa15f5d24a9a833beadc09242227'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:30.572398 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '463caa15f5d24a9a833beadc09242227'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:30.615421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc62093-dcde-40c0-ba0e-f10d1c201e72 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:30.620165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d657c7e-44b7-4536-90cd-8c5332f778b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:30.620800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d657c7e-44b7-4536-90cd-8c5332f778b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.621094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d657c7e-44b7-4536-90cd-8c5332f778b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.621420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d657c7e-44b7-4536-90cd-8c5332f778b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.622138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d657c7e-44b7-4536-90cd-8c5332f778b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:30.626264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d657c7e-44b7-4536-90cd-8c5332f778b8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:30.643122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '463caa15f5d24a9a833beadc09242227'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:30.643792 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9ba45bef-9d48-4ee5-a5da-cf017a6893b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '463caa15f5d24a9a833beadc09242227'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:30.650437 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 953/1934] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:30 2022] DELETE /identity/v3/users/463caa15f5d24a9a833beadc09242227 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:30.654983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415758e-e69f-4c61-b4f3-3566ee71eae6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.655247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415758e-e69f-4c61-b4f3-3566ee71eae6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.655602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415758e-e69f-4c61-b4f3-3566ee71eae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.680682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0415758e-e69f-4c61-b4f3-3566ee71eae6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:30.682043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0415758e-e69f-4c61-b4f3-3566ee71eae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:30.733295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:30.743024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 954/1935] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:30.753891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415758e-e69f-4c61-b4f3-3566ee71eae6 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:30.755300 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0415758e-e69f-4c61-b4f3-3566ee71eae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:30.759424 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 982/1936] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:30 2022] POST /identity/v3/projects => generated 367 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:30.763208 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d657c7e-44b7-4536-90cd-8c5332f778b8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:30.764815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9d657c7e-44b7-4536-90cd-8c5332f778b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:30.823896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415758e-e69f-4c61-b4f3-3566ee71eae6 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:30.836652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d657c7e-44b7-4536-90cd-8c5332f778b8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:30.842105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a09fea-4ce9-47b5-9ce9-acc390af0deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:30.843462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a09fea-4ce9-47b5-9ce9-acc390af0deb admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.844426 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a09fea-4ce9-47b5-9ce9-acc390af0deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.844751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a09fea-4ce9-47b5-9ce9-acc390af0deb admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.845488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a09fea-4ce9-47b5-9ce9-acc390af0deb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:30.847561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a09fea-4ce9-47b5-9ce9-acc390af0deb admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:30.880543 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abcf28-8f93-4a6e-855d-d85c0aaa3637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:30.881164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abcf28-8f93-4a6e-855d-d85c0aaa3637 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.881696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abcf28-8f93-4a6e-855d-d85c0aaa3637 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.881995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abcf28-8f93-4a6e-855d-d85c0aaa3637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.892451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abcf28-8f93-4a6e-855d-d85c0aaa3637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:30.893837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abcf28-8f93-4a6e-855d-d85c0aaa3637 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:30.905174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 955/1937] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:30.937355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 983/1938] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:30 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:30.940492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abcf28-8f93-4a6e-855d-d85c0aaa3637 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:30.950863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abcf28-8f93-4a6e-855d-d85c0aaa3637 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:30.963373 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d2b98-4365-4e44-898e-90808e795ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:30.963893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d2b98-4365-4e44-898e-90808e795ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.964247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d2b98-4365-4e44-898e-90808e795ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.964605 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d2b98-4365-4e44-898e-90808e795ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.965721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d2b98-4365-4e44-898e-90808e795ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:30.967409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d2b98-4365-4e44-898e-90808e795ca3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:30.985597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 956/1939] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:30.988341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a09fea-4ce9-47b5-9ce9-acc390af0deb admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:30.998647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a09fea-4ce9-47b5-9ce9-acc390af0deb admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:31.004041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf09dc5-8739-4604-9891-fcad0309c0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:31.004764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf09dc5-8739-4604-9891-fcad0309c0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:31.005055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf09dc5-8739-4604-9891-fcad0309c0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:31.005385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf09dc5-8739-4604-9891-fcad0309c0fd admin admin] PATH_INFO: `/v3/projects/9e85424657d542b180e7a2b993d78b16/users/f3144b0eda62458ab0d32e112db68594/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:31.021500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf09dc5-8739-4604-9891-fcad0309c0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e85424657d542b180e7a2b993d78b16, user_id=f3144b0eda62458ab0d32e112db68594, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:31.024179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf09dc5-8739-4604-9891-fcad0309c0fd admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:31.135498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 984/1940] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:30 2022] PUT /identity/v3/projects/9e85424657d542b180e7a2b993d78b16/users/f3144b0eda62458ab0d32e112db68594/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:31.139541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d2b98-4365-4e44-898e-90808e795ca3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:31.151021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d2b98-4365-4e44-898e-90808e795ca3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:31.226698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68f379-9d7a-468b-ae54-70df29f4c0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:31.227344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f379-9d7a-468b-ae54-70df29f4c0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:31.227634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f379-9d7a-468b-ae54-70df29f4c0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:31.227914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f379-9d7a-468b-ae54-70df29f4c0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:31.228622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68f379-9d7a-468b-ae54-70df29f4c0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:31.230613 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68f379-9d7a-468b-ae54-70df29f4c0d7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:31.242203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 957/1941] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:29:31.246420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf09dc5-8739-4604-9891-fcad0309c0fd admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:31.256962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf09dc5-8739-4604-9891-fcad0309c0fd admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:31.262501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6fb8d-b162-4f78-93be-47db5dd7b1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:31.263126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6fb8d-b162-4f78-93be-47db5dd7b1a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:31.263410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6fb8d-b162-4f78-93be-47db5dd7b1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:31.264319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6fb8d-b162-4f78-93be-47db5dd7b1a5 admin admin] PATH_INFO: `/v3/projects/9e85424657d542b180e7a2b993d78b16/users/f3144b0eda62458ab0d32e112db68594/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:31.266442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6fb8d-b162-4f78-93be-47db5dd7b1a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e85424657d542b180e7a2b993d78b16, user_id=f3144b0eda62458ab0d32e112db68594, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:31.269064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6fb8d-b162-4f78-93be-47db5dd7b1a5 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:31.278292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:31.280423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 985/1942] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:31 2022] PUT /identity/v3/projects/9e85424657d542b180e7a2b993d78b16/users/f3144b0eda62458ab0d32e112db68594/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:31.292616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbc674-8717-4c6b-b2a0-362e957d0cbe None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:31.292901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbc674-8717-4c6b-b2a0-362e957d0cbe None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:31.293179 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbc674-8717-4c6b-b2a0-362e957d0cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:31.326461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-94dbc674-8717-4c6b-b2a0-362e957d0cbe None None] MFA Rules not processed for user `f3144b0eda62458ab0d32e112db68594`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:31.328223 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-94dbc674-8717-4c6b-b2a0-362e957d0cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:31.374066 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:31.419596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 958/1943] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:31.435205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6fb8d-b162-4f78-93be-47db5dd7b1a5 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:31.437115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e5e6fb8d-b162-4f78-93be-47db5dd7b1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:31.496306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6fb8d-b162-4f78-93be-47db5dd7b1a5 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:31.500903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca84379-c04d-4b2c-bd06-7ea5b50ce7cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:31.502027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca84379-c04d-4b2c-bd06-7ea5b50ce7cd service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:31.502324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca84379-c04d-4b2c-bd06-7ea5b50ce7cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:31.502610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca84379-c04d-4b2c-bd06-7ea5b50ce7cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:31.502887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbc674-8717-4c6b-b2a0-362e957d0cbe None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:31.504356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-94dbc674-8717-4c6b-b2a0-362e957d0cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:31.514013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca84379-c04d-4b2c-bd06-7ea5b50ce7cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:31.516404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca84379-c04d-4b2c-bd06-7ea5b50ce7cd service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:31.529584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 986/1944] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:31.581051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbc674-8717-4c6b-b2a0-362e957d0cbe None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:31.586070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:31.586724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:31.587014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:31.587296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] PATH_INFO: `/v3/projects/9ddc89f69fae4de3979e73c78e222257` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:31.588467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ddc89f69fae4de3979e73c78e222257)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:31.590788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:31.695695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ddc89f69fae4de3979e73c78e222257', 'request_id': 'req-132586e0-20da-44f8-8a14-a1de99827fb4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:31.701552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ddc89f69fae4de3979e73c78e222257', 'request_id': 'req-132586e0-20da-44f8-8a14-a1de99827fb4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:31.710104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ddc89f69fae4de3979e73c78e222257', 'request_id': 'req-132586e0-20da-44f8-8a14-a1de99827fb4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:31.819478 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] The token cache is being invalidate because project 9ddc89f69fae4de3979e73c78e222257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:31.820188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:31.820534 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:31.822126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 959/1945] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:31 2022] DELETE /identity/v3/projects/9ddc89f69fae4de3979e73c78e222257 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:31.826202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca84379-c04d-4b2c-bd06-7ea5b50ce7cd service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:31.837521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca84379-c04d-4b2c-bd06-7ea5b50ce7cd service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:31.914696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:31.914696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:31.915630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:31.915630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] PATH_INFO: `/v3/users/e0b1991aa99a4a3f8cb6b049c51b62de` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:31.919061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b1991aa99a4a3f8cb6b049c51b62de)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:31.921198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:32.105355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b1991aa99a4a3f8cb6b049c51b62de'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:32.118779 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b1991aa99a4a3f8cb6b049c51b62de'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:32.119630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b1991aa99a4a3f8cb6b049c51b62de'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:32.133393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b1991aa99a4a3f8cb6b049c51b62de'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:32.208516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b1991aa99a4a3f8cb6b049c51b62de'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:32.215065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 987/1946] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:31 2022] DELETE /identity/v3/users/e0b1991aa99a4a3f8cb6b049c51b62de => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:32.795375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:32.796384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:32.815210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132586e0-20da-44f8-8a14-a1de99827fb4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:32.878287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:32.884105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:32.884761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:32.885042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:32.885344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] PATH_INFO: `/v3/projects/8606627b74f44945aad4c8f3f167e6f6` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:32.886506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8606627b74f44945aad4c8f3f167e6f6)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:32.888414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:33.043200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8606627b74f44945aad4c8f3f167e6f6', 'request_id': 'req-efc5f752-7158-4171-9e50-848d7b5bc3a9'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:33.049022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8606627b74f44945aad4c8f3f167e6f6', 'request_id': 'req-efc5f752-7158-4171-9e50-848d7b5bc3a9'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:33.056388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8606627b74f44945aad4c8f3f167e6f6', 'request_id': 'req-efc5f752-7158-4171-9e50-848d7b5bc3a9'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:33.163031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] The token cache is being invalidate because project 8606627b74f44945aad4c8f3f167e6f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:33.163345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:33.164126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:33.168913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 960/1947] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:32 2022] DELETE /identity/v3/projects/8606627b74f44945aad4c8f3f167e6f6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:33.170412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:33.173630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:33.189373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee24746-b0f4-46ae-a793-ea534aa67ba3 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:33.212965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:33.263402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:33.265178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:33.265579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:33.265923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] PATH_INFO: `/v3/users/d12b7e808fc04c0e867d22b49b8f5312` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:33.267131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d12b7e808fc04c0e867d22b49b8f5312)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:33.270078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:33.471442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd12b7e808fc04c0e867d22b49b8f5312'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:33.485527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd12b7e808fc04c0e867d22b49b8f5312'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:33.486275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd12b7e808fc04c0e867d22b49b8f5312'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:33.493467 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd12b7e808fc04c0e867d22b49b8f5312'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:33.568212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-333e127f-c9b7-4d81-98d0-5a3b3b66ecb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd12b7e808fc04c0e867d22b49b8f5312'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:33.575183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 988/1948] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:33 2022] DELETE /identity/v3/users/d12b7e808fc04c0e867d22b49b8f5312 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:34.079060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:34.080563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:34.095656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5f752-7158-4171-9e50-848d7b5bc3a9 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:34.175746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:34.176588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.176886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.177171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] PATH_INFO: `/v3/projects/f954555afb2e47e7b546db5b8608db7c` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.178480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f954555afb2e47e7b546db5b8608db7c)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:34.180428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:34.359136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f954555afb2e47e7b546db5b8608db7c', 'request_id': 'req-1c0c7222-726c-483b-b3be-ba4c9c592610'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:34.366804 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f954555afb2e47e7b546db5b8608db7c', 'request_id': 'req-1c0c7222-726c-483b-b3be-ba4c9c592610'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:34.377701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f954555afb2e47e7b546db5b8608db7c', 'request_id': 'req-1c0c7222-726c-483b-b3be-ba4c9c592610'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:34.446075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] The token cache is being invalidate because project f954555afb2e47e7b546db5b8608db7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:34.446504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:34.447288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1c0c7222-726c-483b-b3be-ba4c9c592610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:34.453770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 961/1949] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:34 2022] DELETE /identity/v3/projects/f954555afb2e47e7b546db5b8608db7c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:34.457019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee26cbc-f075-4620-977b-8f2c21a17dae None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.457542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee26cbc-f075-4620-977b-8f2c21a17dae None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.457655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee26cbc-f075-4620-977b-8f2c21a17dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.485439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-aee26cbc-f075-4620-977b-8f2c21a17dae None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:34.485439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-aee26cbc-f075-4620-977b-8f2c21a17dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:34.574902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 989/1950] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:34.577535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9b578-a27e-4442-95a2-9329b765ac3b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.577849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9b578-a27e-4442-95a2-9329b765ac3b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.578565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9b578-a27e-4442-95a2-9329b765ac3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.600311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fef9b578-a27e-4442-95a2-9329b765ac3b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:34.601616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fef9b578-a27e-4442-95a2-9329b765ac3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:34.611044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 962/1951] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:34.615554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee26cbc-f075-4620-977b-8f2c21a17dae None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:34.626449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee26cbc-f075-4620-977b-8f2c21a17dae None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:34.631233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841bcf8-2a77-426a-8c15-1adbe299494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:34.631870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841bcf8-2a77-426a-8c15-1adbe299494f admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.632164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841bcf8-2a77-426a-8c15-1adbe299494f admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.632449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841bcf8-2a77-426a-8c15-1adbe299494f admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.633156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841bcf8-2a77-426a-8c15-1adbe299494f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:29:34.633499 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841bcf8-2a77-426a-8c15-1adbe299494f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:34.635238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841bcf8-2a77-426a-8c15-1adbe299494f admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:34.649988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 990/1952] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:29:34.652205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9b578-a27e-4442-95a2-9329b765ac3b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:34.662767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9b578-a27e-4442-95a2-9329b765ac3b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:34.667509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f9f29-7b41-4d3d-833d-ee57905d0494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:34.668135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f9f29-7b41-4d3d-833d-ee57905d0494 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.668421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f9f29-7b41-4d3d-833d-ee57905d0494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.668701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f9f29-7b41-4d3d-833d-ee57905d0494 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.669423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f9f29-7b41-4d3d-833d-ee57905d0494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:34.672361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f9f29-7b41-4d3d-833d-ee57905d0494 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:34.684684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:34.798843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 963/1953] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:34 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:34.803303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841bcf8-2a77-426a-8c15-1adbe299494f admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:34.815091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841bcf8-2a77-426a-8c15-1adbe299494f admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:34.848020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:34.879597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f9f29-7b41-4d3d-833d-ee57905d0494 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:34.881684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6e5f9f29-7b41-4d3d-833d-ee57905d0494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:34.887738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095d514-bb6e-46f5-822b-6bf0338978c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:34.888437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095d514-bb6e-46f5-822b-6bf0338978c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.892134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095d514-bb6e-46f5-822b-6bf0338978c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.892134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095d514-bb6e-46f5-822b-6bf0338978c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.892134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095d514-bb6e-46f5-822b-6bf0338978c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:34.892829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095d514-bb6e-46f5-822b-6bf0338978c1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:34.951982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f9f29-7b41-4d3d-833d-ee57905d0494 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:34.957424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:34.958105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.958391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.958687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] PATH_INFO: `/v3/users/64771ac5aa624027bd3bdc7c32bf8228` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.959867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] RBAC: Authorizing `identity:delete_user(user_id=64771ac5aa624027bd3bdc7c32bf8228)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:34.961838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:34.976062 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 991/1954] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:34 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:34.980365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095d514-bb6e-46f5-822b-6bf0338978c1 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:34.994322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095d514-bb6e-46f5-822b-6bf0338978c1 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:34.998999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ba3b2-3a50-4726-bd4c-aaac8b1a08b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:34.999807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ba3b2-3a50-4726-bd4c-aaac8b1a08b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.000093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ba3b2-3a50-4726-bd4c-aaac8b1a08b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.000376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ba3b2-3a50-4726-bd4c-aaac8b1a08b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.001079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ba3b2-3a50-4726-bd4c-aaac8b1a08b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:35.002710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ba3b2-3a50-4726-bd4c-aaac8b1a08b9 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:35.017748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 992/1955] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:35.018919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ba3b2-3a50-4726-bd4c-aaac8b1a08b9 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:35.029220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ba3b2-3a50-4726-bd4c-aaac8b1a08b9 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:35.034918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515af7e-3fed-4ba1-8abf-1f8595fcb773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:35.035669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515af7e-3fed-4ba1-8abf-1f8595fcb773 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.035983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515af7e-3fed-4ba1-8abf-1f8595fcb773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.036270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515af7e-3fed-4ba1-8abf-1f8595fcb773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.037343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515af7e-3fed-4ba1-8abf-1f8595fcb773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:35.038959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515af7e-3fed-4ba1-8abf-1f8595fcb773 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:35.050712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 993/1956] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:29:35.054609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515af7e-3fed-4ba1-8abf-1f8595fcb773 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:35.065394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515af7e-3fed-4ba1-8abf-1f8595fcb773 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:35.070113 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd75ca0-ec5f-4376-a3ee-f5d502e29da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:35.070781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75ca0-ec5f-4376-a3ee-f5d502e29da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.071069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75ca0-ec5f-4376-a3ee-f5d502e29da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.071349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75ca0-ec5f-4376-a3ee-f5d502e29da8 admin admin] PATH_INFO: `/v3/projects/4b90af07929741668c5d2eb24687433a/users/8b5de06360314d57b9b31a5cdbdf0d3f/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.087779 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd75ca0-ec5f-4376-a3ee-f5d502e29da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b90af07929741668c5d2eb24687433a, user_id=8b5de06360314d57b9b31a5cdbdf0d3f, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:35.090751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd75ca0-ec5f-4376-a3ee-f5d502e29da8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:35.156386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64771ac5aa624027bd3bdc7c32bf8228'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:35.159936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 994/1957] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:35 2022] PUT /identity/v3/projects/4b90af07929741668c5d2eb24687433a/users/8b5de06360314d57b9b31a5cdbdf0d3f/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:35.169975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64771ac5aa624027bd3bdc7c32bf8228'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:35.171409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ec98-7ec2-4d8e-91ab-a04f6ba93772 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.171701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ec98-7ec2-4d8e-91ab-a04f6ba93772 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.174611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ec98-7ec2-4d8e-91ab-a04f6ba93772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.183563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64771ac5aa624027bd3bdc7c32bf8228'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:35.204512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e808ec98-7ec2-4d8e-91ab-a04f6ba93772 None None] MFA Rules not processed for user `8b5de06360314d57b9b31a5cdbdf0d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:35.205926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e808ec98-7ec2-4d8e-91ab-a04f6ba93772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:35.266639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64771ac5aa624027bd3bdc7c32bf8228'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:35.267659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2c908450-8a36-4dea-b4d2-c2d2f90900db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64771ac5aa624027bd3bdc7c32bf8228'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:35.275699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 964/1958] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:34 2022] DELETE /identity/v3/users/64771ac5aa624027bd3bdc7c32bf8228 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:35.281797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11854ace-c6d6-4ca1-a17e-b2d3b958aed5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.282171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11854ace-c6d6-4ca1-a17e-b2d3b958aed5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.282527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11854ace-c6d6-4ca1-a17e-b2d3b958aed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.287360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 995/1959] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:35.291807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808ec98-7ec2-4d8e-91ab-a04f6ba93772 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:35.293485 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e808ec98-7ec2-4d8e-91ab-a04f6ba93772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:35.307435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-11854ace-c6d6-4ca1-a17e-b2d3b958aed5 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:35.309819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-11854ace-c6d6-4ca1-a17e-b2d3b958aed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:35.372613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808ec98-7ec2-4d8e-91ab-a04f6ba93772 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:35.379112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:35.402034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 965/1960] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:35.412473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11854ace-c6d6-4ca1-a17e-b2d3b958aed5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:35.414145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-11854ace-c6d6-4ca1-a17e-b2d3b958aed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:35.427671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:35.440058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3c37a-61aa-4b9d-b72d-8a5678d6d74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:35.440724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3c37a-61aa-4b9d-b72d-8a5678d6d74b admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.440993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3c37a-61aa-4b9d-b72d-8a5678d6d74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.441618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3c37a-61aa-4b9d-b72d-8a5678d6d74b admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.442572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3c37a-61aa-4b9d-b72d-8a5678d6d74b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:35.444955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3c37a-61aa-4b9d-b72d-8a5678d6d74b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:35.478045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11854ace-c6d6-4ca1-a17e-b2d3b958aed5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:35.481811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19add6-0eac-4b0e-b984-c2f347e16ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:35.482475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19add6-0eac-4b0e-b984-c2f347e16ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.482760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19add6-0eac-4b0e-b984-c2f347e16ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.483042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19add6-0eac-4b0e-b984-c2f347e16ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.492779 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19add6-0eac-4b0e-b984-c2f347e16ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:35.494310 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19add6-0eac-4b0e-b984-c2f347e16ea8 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:35.505071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 966/1961] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:35.513864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 996/1962] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:35 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:35.519316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19add6-0eac-4b0e-b984-c2f347e16ea8 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:35.529521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19add6-0eac-4b0e-b984-c2f347e16ea8 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:35.601118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301f729-df2c-42ff-bf7e-78d821e8a0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:35.603194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301f729-df2c-42ff-bf7e-78d821e8a0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.603194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301f729-df2c-42ff-bf7e-78d821e8a0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.603363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301f729-df2c-42ff-bf7e-78d821e8a0a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.603957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301f729-df2c-42ff-bf7e-78d821e8a0a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:35.606190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301f729-df2c-42ff-bf7e-78d821e8a0a3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:35.700881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 967/1963] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:35 2022] POST /identity/v3/users => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:35.705770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3c37a-61aa-4b9d-b72d-8a5678d6d74b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:35.720059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3c37a-61aa-4b9d-b72d-8a5678d6d74b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:35.725688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdb312-d476-430b-8d90-96f10850d945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:35.726460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdb312-d476-430b-8d90-96f10850d945 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.726921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdb312-d476-430b-8d90-96f10850d945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.727448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdb312-d476-430b-8d90-96f10850d945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.729628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdb312-d476-430b-8d90-96f10850d945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:35.731351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdb312-d476-430b-8d90-96f10850d945 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:35.743834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 997/1964] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:29:35.748200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301f729-df2c-42ff-bf7e-78d821e8a0a3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:35.765444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301f729-df2c-42ff-bf7e-78d821e8a0a3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:35.771157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd42436-bd4a-45c7-b21c-cd017d9b1ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:35.771786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd42436-bd4a-45c7-b21c-cd017d9b1ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.772072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd42436-bd4a-45c7-b21c-cd017d9b1ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.772356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd42436-bd4a-45c7-b21c-cd017d9b1ba5 admin admin] PATH_INFO: `/v3/projects/4e73f4068aa14a9ead482a6629709e73/users/57ab0ddc65e4464b9c40e96381816f18/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.796291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd42436-bd4a-45c7-b21c-cd017d9b1ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e73f4068aa14a9ead482a6629709e73, user_id=57ab0ddc65e4464b9c40e96381816f18, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:35.799580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd42436-bd4a-45c7-b21c-cd017d9b1ba5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:35.861454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 968/1965] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:35 2022] PUT /identity/v3/projects/4e73f4068aa14a9ead482a6629709e73/users/57ab0ddc65e4464b9c40e96381816f18/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:35.865548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdb312-d476-430b-8d90-96f10850d945 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:35.882105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdb312-d476-430b-8d90-96f10850d945 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:35.955577 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd42436-bd4a-45c7-b21c-cd017d9b1ba5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:35.957655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4fd42436-bd4a-45c7-b21c-cd017d9b1ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:35.975643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789ba99-c5e5-4ea5-8cec-fc9629c1fa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:35.976306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ba99-c5e5-4ea5-8cec-fc9629c1fa30 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:35.976598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ba99-c5e5-4ea5-8cec-fc9629c1fa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:35.976884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ba99-c5e5-4ea5-8cec-fc9629c1fa30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:35.977745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789ba99-c5e5-4ea5-8cec-fc9629c1fa30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:35.979409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789ba99-c5e5-4ea5-8cec-fc9629c1fa30 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:35.993205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 998/1966] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:35.997269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789ba99-c5e5-4ea5-8cec-fc9629c1fa30 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:36.008650 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789ba99-c5e5-4ea5-8cec-fc9629c1fa30 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:36.018310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157733c-69e5-4b81-904e-4101321155bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:36.018682 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157733c-69e5-4b81-904e-4101321155bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.019251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157733c-69e5-4b81-904e-4101321155bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.019517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157733c-69e5-4b81-904e-4101321155bc admin admin] PATH_INFO: `/v3/projects/4e73f4068aa14a9ead482a6629709e73/users/57ab0ddc65e4464b9c40e96381816f18/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.021703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157733c-69e5-4b81-904e-4101321155bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e73f4068aa14a9ead482a6629709e73, user_id=57ab0ddc65e4464b9c40e96381816f18, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:36.024576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157733c-69e5-4b81-904e-4101321155bc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:36.036643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 999/1967] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:35 2022] PUT /identity/v3/projects/4e73f4068aa14a9ead482a6629709e73/users/57ab0ddc65e4464b9c40e96381816f18/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:36.050237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5df96-2ed0-4add-87fb-99bfb2033d6a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.050237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5df96-2ed0-4add-87fb-99bfb2033d6a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.050237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5df96-2ed0-4add-87fb-99bfb2033d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.053596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd42436-bd4a-45c7-b21c-cd017d9b1ba5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:36.076867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:36.084838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-39d5df96-2ed0-4add-87fb-99bfb2033d6a None None] MFA Rules not processed for user `57ab0ddc65e4464b9c40e96381816f18`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:36.086509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-39d5df96-2ed0-4add-87fb-99bfb2033d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:36.129444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:36.130191 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.130483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.130766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] PATH_INFO: `/v3/projects/99b364d50dbb463886c0430482d3b67c` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.132106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b364d50dbb463886c0430482d3b67c)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:36.134191 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:36.155209 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:36.175362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1000/1968] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:36.186293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5df96-2ed0-4add-87fb-99bfb2033d6a None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:36.191275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-39d5df96-2ed0-4add-87fb-99bfb2033d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:36.252717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5df96-2ed0-4add-87fb-99bfb2033d6a None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:36.258039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125d595-79fa-4c20-8d03-9498e7534645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:36.258761 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125d595-79fa-4c20-8d03-9498e7534645 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.259050 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125d595-79fa-4c20-8d03-9498e7534645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.259353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125d595-79fa-4c20-8d03-9498e7534645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.269827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125d595-79fa-4c20-8d03-9498e7534645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:36.271273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125d595-79fa-4c20-8d03-9498e7534645 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:36.286318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1001/1969] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:29:36.296226 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b364d50dbb463886c0430482d3b67c', 'request_id': 'req-db86e8d9-0b63-456e-9f48-ddf7596e7055'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:36.302571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b364d50dbb463886c0430482d3b67c', 'request_id': 'req-db86e8d9-0b63-456e-9f48-ddf7596e7055'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:36.314213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b364d50dbb463886c0430482d3b67c', 'request_id': 'req-db86e8d9-0b63-456e-9f48-ddf7596e7055'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:36.418532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] The token cache is being invalidate because project 99b364d50dbb463886c0430482d3b67c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:36.418988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:36.419768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:36.422376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 969/1970] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:35 2022] DELETE /identity/v3/projects/99b364d50dbb463886c0430482d3b67c => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:36.424694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125d595-79fa-4c20-8d03-9498e7534645 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:36.430297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:36.432052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:36.435211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125d595-79fa-4c20-8d03-9498e7534645 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:36.527964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86e8d9-0b63-456e-9f48-ddf7596e7055 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:36.533683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:36.534231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f4661-ec2d-4f70-b446-bf9162fb3b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:36.534726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.534726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.534940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] PATH_INFO: `/v3/users/f3144b0eda62458ab0d32e112db68594` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.535108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f4661-ec2d-4f70-b446-bf9162fb3b56 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.535461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f4661-ec2d-4f70-b446-bf9162fb3b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.535809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f4661-ec2d-4f70-b446-bf9162fb3b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.536093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3144b0eda62458ab0d32e112db68594)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:36.537944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-576f4661-ec2d-4f70-b446-bf9162fb3b56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:36.538120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:36.610252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f4661-ec2d-4f70-b446-bf9162fb3b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:36.611717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f4661-ec2d-4f70-b446-bf9162fb3b56 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:36.622438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1002/1971] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:36.707907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3144b0eda62458ab0d32e112db68594'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:36.714901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:36.721670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3144b0eda62458ab0d32e112db68594'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:36.729169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3144b0eda62458ab0d32e112db68594'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:36.773812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3144b0eda62458ab0d32e112db68594'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:36.774533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a87214e7-08d4-42bd-a2a9-b7c08464f6d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3144b0eda62458ab0d32e112db68594'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:36.782743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 970/1972] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:36 2022] DELETE /identity/v3/users/f3144b0eda62458ab0d32e112db68594 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:37.229349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f4661-ec2d-4f70-b446-bf9162fb3b56 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:37.235526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f4661-ec2d-4f70-b446-bf9162fb3b56 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:37.299379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:37.311920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:37.312576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:37.312973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:37.313145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] PATH_INFO: `/v3/projects/9e85424657d542b180e7a2b993d78b16` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:37.314366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e85424657d542b180e7a2b993d78b16)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:37.316274 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:37.495960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e85424657d542b180e7a2b993d78b16', 'request_id': 'req-69d8dee8-ccc7-4473-bd46-4fb1017c5692'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:37.503319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e85424657d542b180e7a2b993d78b16', 'request_id': 'req-69d8dee8-ccc7-4473-bd46-4fb1017c5692'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:37.602255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e85424657d542b180e7a2b993d78b16', 'request_id': 'req-69d8dee8-ccc7-4473-bd46-4fb1017c5692'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:37.615022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] The token cache is being invalidate because project 9e85424657d542b180e7a2b993d78b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:37.616129 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:37.618250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-69d8dee8-ccc7-4473-bd46-4fb1017c5692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:37.619001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1003/1973] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:37 2022] DELETE /identity/v3/projects/9e85424657d542b180e7a2b993d78b16 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:37.624864 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f3560-128a-4fdc-afe9-bfb4d92194fa None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:37.625442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f3560-128a-4fdc-afe9-bfb4d92194fa None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:37.625442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f3560-128a-4fdc-afe9-bfb4d92194fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:37.648364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-104f3560-128a-4fdc-afe9-bfb4d92194fa None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:37.652944 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-104f3560-128a-4fdc-afe9-bfb4d92194fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:37.719798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 971/1974] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:37.727405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74e2c-4a15-4361-8595-5e842d71c7c7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:37.727709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74e2c-4a15-4361-8595-5e842d71c7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:37.728020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74e2c-4a15-4361-8595-5e842d71c7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:37.749447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-3dd74e2c-4a15-4361-8595-5e842d71c7c7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:37.750683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3dd74e2c-4a15-4361-8595-5e842d71c7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:37.760079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1004/1975] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:37.765285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f3560-128a-4fdc-afe9-bfb4d92194fa None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:37.775610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f3560-128a-4fdc-afe9-bfb4d92194fa None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:37.781461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de20e7b-a030-4e1e-9202-ccc475410854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:37.782145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de20e7b-a030-4e1e-9202-ccc475410854 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:37.782438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de20e7b-a030-4e1e-9202-ccc475410854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:37.782720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de20e7b-a030-4e1e-9202-ccc475410854 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:37.783548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de20e7b-a030-4e1e-9202-ccc475410854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:29:37.783859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de20e7b-a030-4e1e-9202-ccc475410854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:37.785571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de20e7b-a030-4e1e-9202-ccc475410854 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:37.798459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 972/1976] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:29:37.803460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd74e2c-4a15-4361-8595-5e842d71c7c7 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:37.814706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd74e2c-4a15-4361-8595-5e842d71c7c7 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:37.822313 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbd0bf-68f1-4d8a-8361-50f872c6615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:37.822960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbd0bf-68f1-4d8a-8361-50f872c6615a admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:37.823283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbd0bf-68f1-4d8a-8361-50f872c6615a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:37.823573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbd0bf-68f1-4d8a-8361-50f872c6615a admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:37.824275 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dbd0bf-68f1-4d8a-8361-50f872c6615a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:37.826272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dbd0bf-68f1-4d8a-8361-50f872c6615a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:37.918511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1005/1977] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:37 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:37.924232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de20e7b-a030-4e1e-9202-ccc475410854 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:37.936051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de20e7b-a030-4e1e-9202-ccc475410854 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:38.002576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffc483-b451-4ef3-97b8-2aeb378338d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:38.003219 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffc483-b451-4ef3-97b8-2aeb378338d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:38.003505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffc483-b451-4ef3-97b8-2aeb378338d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:38.003788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffc483-b451-4ef3-97b8-2aeb378338d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:38.005455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffc483-b451-4ef3-97b8-2aeb378338d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:38.006530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffc483-b451-4ef3-97b8-2aeb378338d2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:38.132012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 973/1978] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:37 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:38.133774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbd0bf-68f1-4d8a-8361-50f872c6615a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:38.146556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbd0bf-68f1-4d8a-8361-50f872c6615a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:38.152114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313143ec-7220-4507-a9e2-31222d5bcbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:38.152757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313143ec-7220-4507-a9e2-31222d5bcbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:38.153046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313143ec-7220-4507-a9e2-31222d5bcbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:38.153352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313143ec-7220-4507-a9e2-31222d5bcbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:38.154078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313143ec-7220-4507-a9e2-31222d5bcbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:38.156887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313143ec-7220-4507-a9e2-31222d5bcbe3 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:38.167894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1006/1979] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:38.172657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffc483-b451-4ef3-97b8-2aeb378338d2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:38.183115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffc483-b451-4ef3-97b8-2aeb378338d2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:38.188968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79971a-6a9f-445e-832d-ca9d4bb76d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:38.189811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79971a-6a9f-445e-832d-ca9d4bb76d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:38.190095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79971a-6a9f-445e-832d-ca9d4bb76d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:38.190371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79971a-6a9f-445e-832d-ca9d4bb76d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:38.191109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79971a-6a9f-445e-832d-ca9d4bb76d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:38.192727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79971a-6a9f-445e-832d-ca9d4bb76d2a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:38.204250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 974/1980] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:38.208845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313143ec-7220-4507-a9e2-31222d5bcbe3 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:38.221782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313143ec-7220-4507-a9e2-31222d5bcbe3 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:38.226551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81480533-f3dc-4b37-9fef-f17b90faae84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:38.227229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81480533-f3dc-4b37-9fef-f17b90faae84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:38.227520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81480533-f3dc-4b37-9fef-f17b90faae84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:38.227815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81480533-f3dc-4b37-9fef-f17b90faae84 admin admin] PATH_INFO: `/v3/projects/7482210a43074a4c8a0db84e7ba3b704/users/ff512722c4134d368652f87fcdb9ce21/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:38.243949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81480533-f3dc-4b37-9fef-f17b90faae84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7482210a43074a4c8a0db84e7ba3b704, user_id=ff512722c4134d368652f87fcdb9ce21, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:38.246819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81480533-f3dc-4b37-9fef-f17b90faae84 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:38.368652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1007/1981] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:38 2022] PUT /identity/v3/projects/7482210a43074a4c8a0db84e7ba3b704/users/ff512722c4134d368652f87fcdb9ce21/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:38.378952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904434d-4506-41b9-837d-0d9e629841c3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:38.379244 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904434d-4506-41b9-837d-0d9e629841c3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:38.379526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904434d-4506-41b9-837d-0d9e629841c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:38.409338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d904434d-4506-41b9-837d-0d9e629841c3 None None] MFA Rules not processed for user `ff512722c4134d368652f87fcdb9ce21`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:38.410570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d904434d-4506-41b9-837d-0d9e629841c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:38.442053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:38.480103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 975/1982] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:38.485714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81480533-f3dc-4b37-9fef-f17b90faae84 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:38.557110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81480533-f3dc-4b37-9fef-f17b90faae84 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:38.562087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a6759-dabc-4303-b7a8-d610b0401fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:38.562725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6759-dabc-4303-b7a8-d610b0401fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:38.563048 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6759-dabc-4303-b7a8-d610b0401fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:38.563350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6759-dabc-4303-b7a8-d610b0401fc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:38.564013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a6759-dabc-4303-b7a8-d610b0401fc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:38.565998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a6759-dabc-4303-b7a8-d610b0401fc1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:38.691281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3e4d-4ff0-4223-b2b5-d95a44e25f33 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:38.691874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3e4d-4ff0-4223-b2b5-d95a44e25f33 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:38.691874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3e4d-4ff0-4223-b2b5-d95a44e25f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:38.695830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:38.703846 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1008/1983] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:38 2022] POST /identity/v3/projects => generated 377 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:38.708040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a6759-dabc-4303-b7a8-d610b0401fc1 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:38.720478 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-672d3e4d-4ff0-4223-b2b5-d95a44e25f33 None None] MFA Rules not processed for user `eb2a7c1097e0421492e8460e9b64a27b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:38.722306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-672d3e4d-4ff0-4223-b2b5-d95a44e25f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:38.792491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a6759-dabc-4303-b7a8-d610b0401fc1 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:38.797780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace405a8-6e59-4a7f-a3f9-fc84e17393eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:38.798411 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace405a8-6e59-4a7f-a3f9-fc84e17393eb admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:38.798702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace405a8-6e59-4a7f-a3f9-fc84e17393eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:38.799657 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace405a8-6e59-4a7f-a3f9-fc84e17393eb admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:38.800461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace405a8-6e59-4a7f-a3f9-fc84e17393eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:38.801618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 976/1984] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:38.802544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace405a8-6e59-4a7f-a3f9-fc84e17393eb admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:38.926555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1009/1985] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:38 2022] POST /identity/v3/users => generated 295 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:38.928264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d3e4d-4ff0-4223-b2b5-d95a44e25f33 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:38.944525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d3e4d-4ff0-4223-b2b5-d95a44e25f33 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:38.950302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96de4fa-416a-4cc0-9b1d-838bebdabb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:38.951136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96de4fa-416a-4cc0-9b1d-838bebdabb34 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:38.951733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96de4fa-416a-4cc0-9b1d-838bebdabb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:38.952044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96de4fa-416a-4cc0-9b1d-838bebdabb34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:38.952911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96de4fa-416a-4cc0-9b1d-838bebdabb34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:38.954572 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96de4fa-416a-4cc0-9b1d-838bebdabb34 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:38.969760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 977/1986] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:38.971312 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace405a8-6e59-4a7f-a3f9-fc84e17393eb admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:38.981831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace405a8-6e59-4a7f-a3f9-fc84e17393eb admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:38.990413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70fba9-66e8-4e98-b80f-9f44f0ee4b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:38.991227 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70fba9-66e8-4e98-b80f-9f44f0ee4b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:38.991586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70fba9-66e8-4e98-b80f-9f44f0ee4b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:38.992225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70fba9-66e8-4e98-b80f-9f44f0ee4b3a admin admin] PATH_INFO: `/v3/projects/3a94a6fa27d547cd9144399d736e2eda/users/9a438defe4ef49bdb32edd794340442b/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:39.011800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70fba9-66e8-4e98-b80f-9f44f0ee4b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a94a6fa27d547cd9144399d736e2eda, user_id=9a438defe4ef49bdb32edd794340442b, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:39.014609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70fba9-66e8-4e98-b80f-9f44f0ee4b3a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:39.115373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1010/1987] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:38 2022] PUT /identity/v3/projects/3a94a6fa27d547cd9144399d736e2eda/users/9a438defe4ef49bdb32edd794340442b/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:39.125934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96de4fa-416a-4cc0-9b1d-838bebdabb34 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:39.140254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96de4fa-416a-4cc0-9b1d-838bebdabb34 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:39.213748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050d382-d1ec-495d-a806-f09189ab59bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:39.216581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d382-d1ec-495d-a806-f09189ab59bd admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:39.216858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d382-d1ec-495d-a806-f09189ab59bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:39.217143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d382-d1ec-495d-a806-f09189ab59bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:39.222005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050d382-d1ec-495d-a806-f09189ab59bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:39.222005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050d382-d1ec-495d-a806-f09189ab59bd admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:39.237390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 978/1988] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:39.238545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70fba9-66e8-4e98-b80f-9f44f0ee4b3a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:39.252434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70fba9-66e8-4e98-b80f-9f44f0ee4b3a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:39.257974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304e5fc-2562-47f0-b3b2-bcf05a1d64b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:39.258814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304e5fc-2562-47f0-b3b2-bcf05a1d64b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:39.259157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304e5fc-2562-47f0-b3b2-bcf05a1d64b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:39.259443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304e5fc-2562-47f0-b3b2-bcf05a1d64b6 admin admin] PATH_INFO: `/v3/projects/3a94a6fa27d547cd9144399d736e2eda/users/9a438defe4ef49bdb32edd794340442b/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:39.261602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304e5fc-2562-47f0-b3b2-bcf05a1d64b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a94a6fa27d547cd9144399d736e2eda, user_id=9a438defe4ef49bdb32edd794340442b, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:39.265400 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304e5fc-2562-47f0-b3b2-bcf05a1d64b6 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:39.289166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1011/1989] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:39 2022] PUT /identity/v3/projects/3a94a6fa27d547cd9144399d736e2eda/users/9a438defe4ef49bdb32edd794340442b/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:39.291482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b92c3e-37c8-45f2-9e5c-718f984f25c2 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:39.291893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b92c3e-37c8-45f2-9e5c-718f984f25c2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:39.292289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b92c3e-37c8-45f2-9e5c-718f984f25c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:39.332592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-12b92c3e-37c8-45f2-9e5c-718f984f25c2 None None] MFA Rules not processed for user `9a438defe4ef49bdb32edd794340442b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:39.334052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-12b92c3e-37c8-45f2-9e5c-718f984f25c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:39.403890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:39.433598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 979/1990] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:39.445936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304e5fc-2562-47f0-b3b2-bcf05a1d64b6 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:39.448723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9304e5fc-2562-47f0-b3b2-bcf05a1d64b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:39.521014 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304e5fc-2562-47f0-b3b2-bcf05a1d64b6 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:39.528443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c752f49-f9c7-4d4c-a3a4-8f31c51745a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:39.529561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c752f49-f9c7-4d4c-a3a4-8f31c51745a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:39.530579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c752f49-f9c7-4d4c-a3a4-8f31c51745a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:39.531362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c752f49-f9c7-4d4c-a3a4-8f31c51745a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:39.597728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c752f49-f9c7-4d4c-a3a4-8f31c51745a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:39.599173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c752f49-f9c7-4d4c-a3a4-8f31c51745a1 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:39.609558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1012/1991] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:29:40.040873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b92c3e-37c8-45f2-9e5c-718f984f25c2 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:40.042602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-12b92c3e-37c8-45f2-9e5c-718f984f25c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:40.123089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b92c3e-37c8-45f2-9e5c-718f984f25c2 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:40.129978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:40.130771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:40.131067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:40.131351 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] PATH_INFO: `/v3/users/8b5de06360314d57b9b31a5cdbdf0d3f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:40.132689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b5de06360314d57b9b31a5cdbdf0d3f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:40.134931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:40.423906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b5de06360314d57b9b31a5cdbdf0d3f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:40.437334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b5de06360314d57b9b31a5cdbdf0d3f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:40.444225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b5de06360314d57b9b31a5cdbdf0d3f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:40.550877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b5de06360314d57b9b31a5cdbdf0d3f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:40.551535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b5de06360314d57b9b31a5cdbdf0d3f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:40.558922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 980/1992] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:40 2022] DELETE /identity/v3/users/8b5de06360314d57b9b31a5cdbdf0d3f => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:40.564779 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4150f-fedf-4371-b211-cbfd069a1414 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:40.566848 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4150f-fedf-4371-b211-cbfd069a1414 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:40.566848 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4150f-fedf-4371-b211-cbfd069a1414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:40.580863 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:40.587137 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e8e4150f-fedf-4371-b211-cbfd069a1414 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:40.588378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e8e4150f-fedf-4371-b211-cbfd069a1414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:40.655793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1013/1993] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:40.665550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:40.667028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:40.736492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a389289-9ee0-4ef8-812f-ef3277a453cb admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:40.740428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842a7bd-fd01-49f3-9526-7727f24fb363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:40.741054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842a7bd-fd01-49f3-9526-7727f24fb363 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:40.741366 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842a7bd-fd01-49f3-9526-7727f24fb363 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:40.741649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842a7bd-fd01-49f3-9526-7727f24fb363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:40.752448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842a7bd-fd01-49f3-9526-7727f24fb363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:40.754022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842a7bd-fd01-49f3-9526-7727f24fb363 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:40.764935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 981/1994] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:41.216303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4150f-fedf-4371-b211-cbfd069a1414 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:41.221203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e8e4150f-fedf-4371-b211-cbfd069a1414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:41.239820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4150f-fedf-4371-b211-cbfd069a1414 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:41.245268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:41.245971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:41.246242 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:41.246622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] PATH_INFO: `/v3/projects/4b90af07929741668c5d2eb24687433a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:41.247870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b90af07929741668c5d2eb24687433a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:41.250624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:41.378639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b90af07929741668c5d2eb24687433a', 'request_id': 'req-694c6a97-ecfe-409b-a8a9-a52c16fcba93'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:41.384669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b90af07929741668c5d2eb24687433a', 'request_id': 'req-694c6a97-ecfe-409b-a8a9-a52c16fcba93'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:41.499135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b90af07929741668c5d2eb24687433a', 'request_id': 'req-694c6a97-ecfe-409b-a8a9-a52c16fcba93'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:41.512082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] The token cache is being invalidate because project 4b90af07929741668c5d2eb24687433a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:41.512438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:41.513279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:41.515134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1014/1995] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:41 2022] DELETE /identity/v3/projects/4b90af07929741668c5d2eb24687433a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:41.519291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842a7bd-fd01-49f3-9526-7727f24fb363 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:41.530832 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842a7bd-fd01-49f3-9526-7727f24fb363 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:41.535117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:41.594525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:41.595105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:41.595389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:41.595666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] PATH_INFO: `/v3/users/57ab0ddc65e4464b9c40e96381816f18` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:41.596963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ab0ddc65e4464b9c40e96381816f18)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:41.599387 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:41.814016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ab0ddc65e4464b9c40e96381816f18'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:41.837810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ab0ddc65e4464b9c40e96381816f18'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:41.850027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ab0ddc65e4464b9c40e96381816f18'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:41.916511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ab0ddc65e4464b9c40e96381816f18'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:41.917591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-99082f09-d157-497a-9b58-bc9746688a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ab0ddc65e4464b9c40e96381816f18'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:41.926685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 982/1996] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:41 2022] DELETE /identity/v3/users/57ab0ddc65e4464b9c40e96381816f18 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:42.380611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:42.382278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:42.408400 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c6a97-ecfe-409b-a8a9-a52c16fcba93 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:42.474514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:42.475342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:42.475736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:42.476141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] PATH_INFO: `/v3/projects/4e73f4068aa14a9ead482a6629709e73` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:42.477727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e73f4068aa14a9ead482a6629709e73)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:42.479919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:42.591885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:42.604893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e73f4068aa14a9ead482a6629709e73', 'request_id': 'req-ffc3e126-0801-4900-98c4-886fb8a9512b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:42.610749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e73f4068aa14a9ead482a6629709e73', 'request_id': 'req-ffc3e126-0801-4900-98c4-886fb8a9512b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:42.671583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e73f4068aa14a9ead482a6629709e73', 'request_id': 'req-ffc3e126-0801-4900-98c4-886fb8a9512b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:42.683782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] The token cache is being invalidate because project 4e73f4068aa14a9ead482a6629709e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:42.684071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:42.684734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ffc3e126-0801-4900-98c4-886fb8a9512b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:42.686985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1015/1997] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:42 2022] DELETE /identity/v3/projects/4e73f4068aa14a9ead482a6629709e73 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:42.691942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc777c-dda5-4d0a-abd1-847cb78a373c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:42.692711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc777c-dda5-4d0a-abd1-847cb78a373c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:42.693006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc777c-dda5-4d0a-abd1-847cb78a373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:42.713889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6cdc777c-dda5-4d0a-abd1-847cb78a373c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:42.715408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6cdc777c-dda5-4d0a-abd1-847cb78a373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:42.778466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 983/1998] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:42.786866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcba0b3-4dfb-4e93-91bb-f18752349d83 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:42.787157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcba0b3-4dfb-4e93-91bb-f18752349d83 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:42.787465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcba0b3-4dfb-4e93-91bb-f18752349d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:42.807498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-6dcba0b3-4dfb-4e93-91bb-f18752349d83 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:42.808712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6dcba0b3-4dfb-4e93-91bb-f18752349d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:42.817446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1016/1999] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:42.821713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc777c-dda5-4d0a-abd1-847cb78a373c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:42.832042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc777c-dda5-4d0a-abd1-847cb78a373c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:42.837709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87dc1c-526f-4ea7-9692-a1018371692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:42.838653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87dc1c-526f-4ea7-9692-a1018371692c admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:42.838941 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87dc1c-526f-4ea7-9692-a1018371692c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:42.839224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87dc1c-526f-4ea7-9692-a1018371692c admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:42.840213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87dc1c-526f-4ea7-9692-a1018371692c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:29:42.840494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87dc1c-526f-4ea7-9692-a1018371692c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:42.842136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87dc1c-526f-4ea7-9692-a1018371692c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:42.856335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 984/2000] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:29:42.861518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcba0b3-4dfb-4e93-91bb-f18752349d83 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:42.871212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcba0b3-4dfb-4e93-91bb-f18752349d83 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:42.876091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5085-a5c1-4ced-a3b4-6c9c466b1c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:42.876709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5085-a5c1-4ced-a3b4-6c9c466b1c20 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:42.876996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5085-a5c1-4ced-a3b4-6c9c466b1c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:42.877278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5085-a5c1-4ced-a3b4-6c9c466b1c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:42.877989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f5085-a5c1-4ced-a3b4-6c9c466b1c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:42.880076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f5085-a5c1-4ced-a3b4-6c9c466b1c20 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:42.964167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1017/2001] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:42 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:42.969426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87dc1c-526f-4ea7-9692-a1018371692c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:42.979578 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87dc1c-526f-4ea7-9692-a1018371692c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:42.988797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:43.037706 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb426f7e-f08b-4b82-a249-535f5fe1c639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:43.038348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb426f7e-f08b-4b82-a249-535f5fe1c639 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:43.038636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb426f7e-f08b-4b82-a249-535f5fe1c639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:43.038923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb426f7e-f08b-4b82-a249-535f5fe1c639 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:43.039614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb426f7e-f08b-4b82-a249-535f5fe1c639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:43.041693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb426f7e-f08b-4b82-a249-535f5fe1c639 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:43.108485 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 985/2002] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:42 2022] POST /identity/v3/users => generated 283 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:29:43.112596 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5085-a5c1-4ced-a3b4-6c9c466b1c20 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:43.128032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5085-a5c1-4ced-a3b4-6c9c466b1c20 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:43.133105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a9cdb-43ff-4d42-87ef-7747258ba1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:43.133773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a9cdb-43ff-4d42-87ef-7747258ba1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:43.134069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a9cdb-43ff-4d42-87ef-7747258ba1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:43.134457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a9cdb-43ff-4d42-87ef-7747258ba1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:43.135052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a9cdb-43ff-4d42-87ef-7747258ba1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:43.136653 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a9cdb-43ff-4d42-87ef-7747258ba1a6 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:43.148730 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1018/2003] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:43.153107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb426f7e-f08b-4b82-a249-535f5fe1c639 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:43.165021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb426f7e-f08b-4b82-a249-535f5fe1c639 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:43.169812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bafda-0948-47fb-9272-d005b718c8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:43.171005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bafda-0948-47fb-9272-d005b718c8dc admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:43.174526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bafda-0948-47fb-9272-d005b718c8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:43.174526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bafda-0948-47fb-9272-d005b718c8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:43.174526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bafda-0948-47fb-9272-d005b718c8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:43.174526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bafda-0948-47fb-9272-d005b718c8dc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:43.188053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 986/2004] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:43.192572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a9cdb-43ff-4d42-87ef-7747258ba1a6 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:43.203292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a9cdb-43ff-4d42-87ef-7747258ba1a6 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:43.209324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63d064-aa65-4fbf-b1e0-4f92450cda66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:43.210014 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63d064-aa65-4fbf-b1e0-4f92450cda66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:43.210306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63d064-aa65-4fbf-b1e0-4f92450cda66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:43.210592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63d064-aa65-4fbf-b1e0-4f92450cda66 admin admin] PATH_INFO: `/v3/projects/64429063f68e4e10a4bca7d0fa5d134e/users/239ccb720d0c424dbcc7dbd8d6d3d336/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:43.236461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63d064-aa65-4fbf-b1e0-4f92450cda66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64429063f68e4e10a4bca7d0fa5d134e, user_id=239ccb720d0c424dbcc7dbd8d6d3d336, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:43.239179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63d064-aa65-4fbf-b1e0-4f92450cda66 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:43.300916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1019/2005] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:43 2022] PUT /identity/v3/projects/64429063f68e4e10a4bca7d0fa5d134e/users/239ccb720d0c424dbcc7dbd8d6d3d336/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:43.317071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4f998-8135-4425-9f82-abc69bd53825 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:43.317404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4f998-8135-4425-9f82-abc69bd53825 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:43.317692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4f998-8135-4425-9f82-abc69bd53825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:43.347871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-10c4f998-8135-4425-9f82-abc69bd53825 None None] MFA Rules not processed for user `239ccb720d0c424dbcc7dbd8d6d3d336`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:43.349847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-10c4f998-8135-4425-9f82-abc69bd53825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:43.425535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 987/2006] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:43.430181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63d064-aa65-4fbf-b1e0-4f92450cda66 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:43.495711 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63d064-aa65-4fbf-b1e0-4f92450cda66 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:43.500938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980792ee-bdd5-4060-9a9c-3b33e688f2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:43.501597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980792ee-bdd5-4060-9a9c-3b33e688f2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:43.501885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980792ee-bdd5-4060-9a9c-3b33e688f2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:43.502200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980792ee-bdd5-4060-9a9c-3b33e688f2f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:43.502886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980792ee-bdd5-4060-9a9c-3b33e688f2f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:43.504802 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980792ee-bdd5-4060-9a9c-3b33e688f2f2 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:43.603773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1020/2007] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:43 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:43.607811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4f998-8135-4425-9f82-abc69bd53825 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:43.621196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4f998-8135-4425-9f82-abc69bd53825 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:43.682000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a8c10-4166-455b-81f5-ddaa870cc210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:43.682638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a8c10-4166-455b-81f5-ddaa870cc210 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:43.682931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a8c10-4166-455b-81f5-ddaa870cc210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:43.683209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a8c10-4166-455b-81f5-ddaa870cc210 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:43.683891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a8c10-4166-455b-81f5-ddaa870cc210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:43.686450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a8c10-4166-455b-81f5-ddaa870cc210 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:43.785879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 988/2008] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:43 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:43.789689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980792ee-bdd5-4060-9a9c-3b33e688f2f2 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:43.800909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980792ee-bdd5-4060-9a9c-3b33e688f2f2 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:43.805695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1117df-a7bf-4549-a74d-d9d1fcb063dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:43.806342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1117df-a7bf-4549-a74d-d9d1fcb063dd admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:43.806625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1117df-a7bf-4549-a74d-d9d1fcb063dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:43.806903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1117df-a7bf-4549-a74d-d9d1fcb063dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:43.807629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1117df-a7bf-4549-a74d-d9d1fcb063dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:43.809231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1117df-a7bf-4549-a74d-d9d1fcb063dd admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:43.820919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1021/2009] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:43.825344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a8c10-4166-455b-81f5-ddaa870cc210 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:43.836593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a8c10-4166-455b-81f5-ddaa870cc210 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:43.841665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd87c3c-a320-4433-a8c1-557f66bb5321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:43.842342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd87c3c-a320-4433-a8c1-557f66bb5321 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:43.842631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd87c3c-a320-4433-a8c1-557f66bb5321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:43.842915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd87c3c-a320-4433-a8c1-557f66bb5321 admin admin] PATH_INFO: `/v3/projects/be4dda9dcc4a46bca20055d00e47445f/users/7be95ff25ea94ed396f038855b5bf71f/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:43.852035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:43.858996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd87c3c-a320-4433-a8c1-557f66bb5321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be4dda9dcc4a46bca20055d00e47445f, user_id=7be95ff25ea94ed396f038855b5bf71f, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:43.862657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd87c3c-a320-4433-a8c1-557f66bb5321 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:43.968459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 989/2010] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:43 2022] PUT /identity/v3/projects/be4dda9dcc4a46bca20055d00e47445f/users/7be95ff25ea94ed396f038855b5bf71f/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:43.972884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1117df-a7bf-4549-a74d-d9d1fcb063dd admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:43.983036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1117df-a7bf-4549-a74d-d9d1fcb063dd admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:44.014099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:44.043897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de87bae-0d25-4fc0-ba9a-e018c547f36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:44.044445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de87bae-0d25-4fc0-ba9a-e018c547f36c admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:44.044732 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de87bae-0d25-4fc0-ba9a-e018c547f36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:44.045021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de87bae-0d25-4fc0-ba9a-e018c547f36c admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:44.045769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de87bae-0d25-4fc0-ba9a-e018c547f36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:44.047380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de87bae-0d25-4fc0-ba9a-e018c547f36c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:44.057873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1022/2011] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:44.062195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd87c3c-a320-4433-a8c1-557f66bb5321 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:44.074215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd87c3c-a320-4433-a8c1-557f66bb5321 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:44.079054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fc53c-f521-4a30-ab2b-8d0c194ea9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:44.079808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fc53c-f521-4a30-ab2b-8d0c194ea9fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:44.080177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fc53c-f521-4a30-ab2b-8d0c194ea9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:44.080296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fc53c-f521-4a30-ab2b-8d0c194ea9fc admin admin] PATH_INFO: `/v3/projects/be4dda9dcc4a46bca20055d00e47445f/users/7be95ff25ea94ed396f038855b5bf71f/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:44.082304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fc53c-f521-4a30-ab2b-8d0c194ea9fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=be4dda9dcc4a46bca20055d00e47445f, user_id=7be95ff25ea94ed396f038855b5bf71f, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:44.085026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fc53c-f521-4a30-ab2b-8d0c194ea9fc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:44.096042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 990/2012] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:44 2022] PUT /identity/v3/projects/be4dda9dcc4a46bca20055d00e47445f/users/7be95ff25ea94ed396f038855b5bf71f/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:44.107325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753cd9c-81e3-400b-819c-507b8da2d045 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:44.107636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753cd9c-81e3-400b-819c-507b8da2d045 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:44.108054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753cd9c-81e3-400b-819c-507b8da2d045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:44.137195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-1753cd9c-81e3-400b-819c-507b8da2d045 None None] MFA Rules not processed for user `7be95ff25ea94ed396f038855b5bf71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:44.138455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1753cd9c-81e3-400b-819c-507b8da2d045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:44.209984 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1023/2013] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:44.222655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fc53c-f521-4a30-ab2b-8d0c194ea9fc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:44.224527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e21fc53c-f521-4a30-ab2b-8d0c194ea9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:44.288776 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fc53c-f521-4a30-ab2b-8d0c194ea9fc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:44.292503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658ec93-f2ed-4384-a2b7-df82907b3449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:44.293095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658ec93-f2ed-4384-a2b7-df82907b3449 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:44.293422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658ec93-f2ed-4384-a2b7-df82907b3449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:44.294610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658ec93-f2ed-4384-a2b7-df82907b3449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:44.306644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658ec93-f2ed-4384-a2b7-df82907b3449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:44.309683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658ec93-f2ed-4384-a2b7-df82907b3449 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:44.325490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 991/2014] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:44.620833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753cd9c-81e3-400b-819c-507b8da2d045 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:44.632243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753cd9c-81e3-400b-819c-507b8da2d045 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:44.637107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcbee7-5df5-4ab3-8c78-47a316dab818 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:44.637107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcbee7-5df5-4ab3-8c78-47a316dab818 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:44.637107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcbee7-5df5-4ab3-8c78-47a316dab818 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:44.637493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcbee7-5df5-4ab3-8c78-47a316dab818 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:44.698696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcbee7-5df5-4ab3-8c78-47a316dab818 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:44.700198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcbee7-5df5-4ab3-8c78-47a316dab818 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:44.709724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1024/2015] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:45.948399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658ec93-f2ed-4384-a2b7-df82907b3449 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:45.958737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658ec93-f2ed-4384-a2b7-df82907b3449 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:45.962985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc6b63-e84d-429f-afcd-6a3cdc4d0428 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:45.963549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc6b63-e84d-429f-afcd-6a3cdc4d0428 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:45.963839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc6b63-e84d-429f-afcd-6a3cdc4d0428 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:45.964127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc6b63-e84d-429f-afcd-6a3cdc4d0428 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:45.966972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1bdc6b63-e84d-429f-afcd-6a3cdc4d0428 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:46.050656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc6b63-e84d-429f-afcd-6a3cdc4d0428 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:46.052119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc6b63-e84d-429f-afcd-6a3cdc4d0428 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:46.062498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 992/2016] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:51.411582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcbee7-5df5-4ab3-8c78-47a316dab818 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:51.463911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:51.476573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcbee7-5df5-4ab3-8c78-47a316dab818 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:51.481277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:51.481952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:51.482240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:51.482524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] PATH_INFO: `/v3/users/239ccb720d0c424dbcc7dbd8d6d3d336` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:51.483696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=239ccb720d0c424dbcc7dbd8d6d3d336)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:51.485668 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:51.696362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '239ccb720d0c424dbcc7dbd8d6d3d336'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:51.708808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '239ccb720d0c424dbcc7dbd8d6d3d336'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:51.710203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '239ccb720d0c424dbcc7dbd8d6d3d336'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:51.716718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '239ccb720d0c424dbcc7dbd8d6d3d336'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:51.767134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '239ccb720d0c424dbcc7dbd8d6d3d336'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:51.775969 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1025/2017] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:51 2022] DELETE /identity/v3/users/239ccb720d0c424dbcc7dbd8d6d3d336 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:51.780596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fac19f-4a81-49bd-9797-526cf52e121c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:51.785035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fac19f-4a81-49bd-9797-526cf52e121c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:51.785035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fac19f-4a81-49bd-9797-526cf52e121c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:51.800947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-00fac19f-4a81-49bd-9797-526cf52e121c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:51.802201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-00fac19f-4a81-49bd-9797-526cf52e121c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:51.862867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 993/2018] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:51.872637 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:51.874337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:51.962201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd7cce-524c-444b-8eec-c6cb0c7c3d35 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:51.966151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fdbee-cc7f-4b22-8cc3-988eb144c37d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:51.966793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fdbee-cc7f-4b22-8cc3-988eb144c37d service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:51.967069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fdbee-cc7f-4b22-8cc3-988eb144c37d service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:51.967346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fdbee-cc7f-4b22-8cc3-988eb144c37d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:51.977603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fdbee-cc7f-4b22-8cc3-988eb144c37d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:51.979055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fdbee-cc7f-4b22-8cc3-988eb144c37d service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:51.988957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1026/2019] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:52.574542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fac19f-4a81-49bd-9797-526cf52e121c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:52.576106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-00fac19f-4a81-49bd-9797-526cf52e121c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:52.591578 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fac19f-4a81-49bd-9797-526cf52e121c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:52.596735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:52.598791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:52.599079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:52.599402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] PATH_INFO: `/v3/projects/64429063f68e4e10a4bca7d0fa5d134e` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:52.601109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] RBAC: Authorizing `identity:delete_project(project_id=64429063f68e4e10a4bca7d0fa5d134e)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:52.602879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:52.613200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:52.785296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64429063f68e4e10a4bca7d0fa5d134e', 'request_id': 'req-e6bd7df4-49a6-40aa-b50f-838b624c957b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:52.790797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64429063f68e4e10a4bca7d0fa5d134e', 'request_id': 'req-e6bd7df4-49a6-40aa-b50f-838b624c957b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:52.797476 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64429063f68e4e10a4bca7d0fa5d134e', 'request_id': 'req-e6bd7df4-49a6-40aa-b50f-838b624c957b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:52.910636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] The token cache is being invalidate because project 64429063f68e4e10a4bca7d0fa5d134e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:52.910932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:52.911579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:52.913142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 994/2020] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:52 2022] DELETE /identity/v3/projects/64429063f68e4e10a4bca7d0fa5d134e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:52.917175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fdbee-cc7f-4b22-8cc3-988eb144c37d service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:52.927613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fdbee-cc7f-4b22-8cc3-988eb144c37d service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:52.989622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:52.990283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:52.991127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:52.991127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] PATH_INFO: `/v3/users/7be95ff25ea94ed396f038855b5bf71f` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:52.992554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be95ff25ea94ed396f038855b5bf71f)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:52.995672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:53.011440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:53.255785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be95ff25ea94ed396f038855b5bf71f'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:53.272830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be95ff25ea94ed396f038855b5bf71f'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:53.274186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be95ff25ea94ed396f038855b5bf71f'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:53.292284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be95ff25ea94ed396f038855b5bf71f'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:53.412742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-03859957-ad50-4c00-acb4-ef6b4abe8f45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be95ff25ea94ed396f038855b5bf71f'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:53.419513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1027/2021] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:29:52 2022] DELETE /identity/v3/users/7be95ff25ea94ed396f038855b5bf71f => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:29:53.940415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:53.941890 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:53.967926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd7df4-49a6-40aa-b50f-838b624c957b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:54.045810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:54.046461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.046752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.047038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] PATH_INFO: `/v3/projects/be4dda9dcc4a46bca20055d00e47445f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.048282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be4dda9dcc4a46bca20055d00e47445f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:54.050601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:54.196583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be4dda9dcc4a46bca20055d00e47445f', 'request_id': 'req-1ce0178a-be82-4fac-9004-0bf2030270a7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:54.202549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be4dda9dcc4a46bca20055d00e47445f', 'request_id': 'req-1ce0178a-be82-4fac-9004-0bf2030270a7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:54.209799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be4dda9dcc4a46bca20055d00e47445f', 'request_id': 'req-1ce0178a-be82-4fac-9004-0bf2030270a7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:54.322186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] The token cache is being invalidate because project be4dda9dcc4a46bca20055d00e47445f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:29:54.322461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:54.323109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ce0178a-be82-4fac-9004-0bf2030270a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:29:54.324661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 995/2022] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:29:53 2022] DELETE /identity/v3/projects/be4dda9dcc4a46bca20055d00e47445f => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:54.338405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568ef0f-113d-4680-849c-05e0fd301b19 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.338405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568ef0f-113d-4680-849c-05e0fd301b19 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.338405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568ef0f-113d-4680-849c-05e0fd301b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.353504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4568ef0f-113d-4680-849c-05e0fd301b19 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:54.354740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4568ef0f-113d-4680-849c-05e0fd301b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:54.420801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1028/2023] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:54.428701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe7634-337c-47c4-8bc1-6b5a2fcb962e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.428988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe7634-337c-47c4-8bc1-6b5a2fcb962e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.429253 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe7634-337c-47c4-8bc1-6b5a2fcb962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.449469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1bbe7634-337c-47c4-8bc1-6b5a2fcb962e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:54.450576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1bbe7634-337c-47c4-8bc1-6b5a2fcb962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:54.459061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 996/2024] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:54.463557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568ef0f-113d-4680-849c-05e0fd301b19 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:54.473627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568ef0f-113d-4680-849c-05e0fd301b19 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:54.478465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef7172-66c3-4743-9f6d-c0f0c6bbd8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:54.479077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef7172-66c3-4743-9f6d-c0f0c6bbd8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.479365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef7172-66c3-4743-9f6d-c0f0c6bbd8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.479641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef7172-66c3-4743-9f6d-c0f0c6bbd8c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.480386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef7172-66c3-4743-9f6d-c0f0c6bbd8c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:29:54.480691 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef7172-66c3-4743-9f6d-c0f0c6bbd8c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:54.482336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef7172-66c3-4743-9f6d-c0f0c6bbd8c2 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:54.494286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1029/2025] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:29:54.498864 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe7634-337c-47c4-8bc1-6b5a2fcb962e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:54.509267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe7634-337c-47c4-8bc1-6b5a2fcb962e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:54.514583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb5e30-2d02-4a09-b5c2-135e1f713aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:54.515221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb5e30-2d02-4a09-b5c2-135e1f713aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.515498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb5e30-2d02-4a09-b5c2-135e1f713aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.515772 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb5e30-2d02-4a09-b5c2-135e1f713aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.516445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb5e30-2d02-4a09-b5c2-135e1f713aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:54.519235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb5e30-2d02-4a09-b5c2-135e1f713aa9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:54.538701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:54.669173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 997/2026] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:29:54 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:54.672039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef7172-66c3-4743-9f6d-c0f0c6bbd8c2 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:54.684178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef7172-66c3-4743-9f6d-c0f0c6bbd8c2 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:54.746645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12db95-6dff-4983-a132-d2c98dcff97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:54.747274 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12db95-6dff-4983-a132-d2c98dcff97e admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.747567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12db95-6dff-4983-a132-d2c98dcff97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.747850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12db95-6dff-4983-a132-d2c98dcff97e admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.748601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12db95-6dff-4983-a132-d2c98dcff97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:54.750667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12db95-6dff-4983-a132-d2c98dcff97e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:54.858729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1030/2027] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:29:54 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:29:54.863731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb5e30-2d02-4a09-b5c2-135e1f713aa9 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:54.874272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb5e30-2d02-4a09-b5c2-135e1f713aa9 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:54.879668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a831e-1c57-4b7a-89f3-062433e0594d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:54.880287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a831e-1c57-4b7a-89f3-062433e0594d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.880603 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a831e-1c57-4b7a-89f3-062433e0594d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.880888 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a831e-1c57-4b7a-89f3-062433e0594d admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.881630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a831e-1c57-4b7a-89f3-062433e0594d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:54.883249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a831e-1c57-4b7a-89f3-062433e0594d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:54.893797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 998/2028] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:54.897911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12db95-6dff-4983-a132-d2c98dcff97e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:54.908488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12db95-6dff-4983-a132-d2c98dcff97e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:54.913811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209df3c7-ea67-418f-b5ce-b9a0386f83c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:54.914449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209df3c7-ea67-418f-b5ce-b9a0386f83c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.914739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209df3c7-ea67-418f-b5ce-b9a0386f83c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.915021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209df3c7-ea67-418f-b5ce-b9a0386f83c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.915789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209df3c7-ea67-418f-b5ce-b9a0386f83c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:54.917573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209df3c7-ea67-418f-b5ce-b9a0386f83c5 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:54.922577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:54.929804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1031/2029] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:29:54.934189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a831e-1c57-4b7a-89f3-062433e0594d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:54.945333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a831e-1c57-4b7a-89f3-062433e0594d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:54.950599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81505e-94af-481b-be89-1b69fe175e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:54.951283 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81505e-94af-481b-be89-1b69fe175e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.951576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81505e-94af-481b-be89-1b69fe175e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.951867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81505e-94af-481b-be89-1b69fe175e14 admin admin] PATH_INFO: `/v3/projects/ee49c49b426648b9a3b5fae15f24bcea/users/a3559dcd910047d8b36a2c347fc1d968/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.968289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81505e-94af-481b-be89-1b69fe175e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee49c49b426648b9a3b5fae15f24bcea, user_id=a3559dcd910047d8b36a2c347fc1d968, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:54.971798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81505e-94af-481b-be89-1b69fe175e14 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:55.053785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 999/2030] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:29:54 2022] PUT /identity/v3/projects/ee49c49b426648b9a3b5fae15f24bcea/users/a3559dcd910047d8b36a2c347fc1d968/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:29:55.063800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e550fd-7ceb-43e3-a041-012e757451fe None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:55.064127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e550fd-7ceb-43e3-a041-012e757451fe None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:55.064498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e550fd-7ceb-43e3-a041-012e757451fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:55.093380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b9e550fd-7ceb-43e3-a041-012e757451fe None None] MFA Rules not processed for user `a3559dcd910047d8b36a2c347fc1d968`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:55.094603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b9e550fd-7ceb-43e3-a041-012e757451fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:55.165695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1032/2031] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:29:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:55.177292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81505e-94af-481b-be89-1b69fe175e14 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:55.179141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ae81505e-94af-481b-be89-1b69fe175e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:55.238282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81505e-94af-481b-be89-1b69fe175e14 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:55.242056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a88d9c-d317-419e-9fb5-7ec17998c11e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:55.242688 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88d9c-d317-419e-9fb5-7ec17998c11e service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:55.243038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88d9c-d317-419e-9fb5-7ec17998c11e service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:55.243324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88d9c-d317-419e-9fb5-7ec17998c11e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:55.252168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a88d9c-d317-419e-9fb5-7ec17998c11e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:55.260321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a88d9c-d317-419e-9fb5-7ec17998c11e service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:55.262619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1000/2032] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:05.162851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=33 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:05.258472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:08.268227 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e550fd-7ceb-43e3-a041-012e757451fe None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:08.346815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e550fd-7ceb-43e3-a041-012e757451fe None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:08.352228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:08.353125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:08.353420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:08.353705 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] PATH_INFO: `/v3/users/a3559dcd910047d8b36a2c347fc1d968` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:08.354946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3559dcd910047d8b36a2c347fc1d968)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:08.356889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:08.598613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3559dcd910047d8b36a2c347fc1d968'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:08.618639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3559dcd910047d8b36a2c347fc1d968'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:08.619313 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3559dcd910047d8b36a2c347fc1d968'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:08.626767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3559dcd910047d8b36a2c347fc1d968'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:08.669522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3559dcd910047d8b36a2c347fc1d968'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:08.676359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1033/2033] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:30:08 2022] DELETE /identity/v3/users/a3559dcd910047d8b36a2c347fc1d968 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:08.682033 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2124c5-e0f1-46bc-a6f3-cec20fb848bf None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:08.682544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2124c5-e0f1-46bc-a6f3-cec20fb848bf None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:08.682635 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2124c5-e0f1-46bc-a6f3-cec20fb848bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:08.702211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ce2124c5-e0f1-46bc-a6f3-cec20fb848bf None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:08.703430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ce2124c5-e0f1-46bc-a6f3-cec20fb848bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:08.777874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1001/2034] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:08.779420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:08.780950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:08.845394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0c64c-9ad4-4cf4-9f49-848fe36b9cea admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:08.849184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e48a9-f302-4dcf-b860-d7feeedb56ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:08.849832 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e48a9-f302-4dcf-b860-d7feeedb56ad service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:08.850122 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e48a9-f302-4dcf-b860-d7feeedb56ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:08.850402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e48a9-f302-4dcf-b860-d7feeedb56ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:08.859687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e48a9-f302-4dcf-b860-d7feeedb56ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:08.861157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e48a9-f302-4dcf-b860-d7feeedb56ad service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:08.871205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1034/2035] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:09.316742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2124c5-e0f1-46bc-a6f3-cec20fb848bf None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:09.318246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ce2124c5-e0f1-46bc-a6f3-cec20fb848bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:09.331645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2124c5-e0f1-46bc-a6f3-cec20fb848bf None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:09.337501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:09.338160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:09.338442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:09.339015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] PATH_INFO: `/v3/projects/ee49c49b426648b9a3b5fae15f24bcea` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:09.340182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee49c49b426648b9a3b5fae15f24bcea)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:09.342133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:09.432466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee49c49b426648b9a3b5fae15f24bcea', 'request_id': 'req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:09.438673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee49c49b426648b9a3b5fae15f24bcea', 'request_id': 'req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:09.448061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee49c49b426648b9a3b5fae15f24bcea', 'request_id': 'req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:09.504338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] The token cache is being invalidate because project ee49c49b426648b9a3b5fae15f24bcea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:30:09.504624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:09.505397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e80d75d1-24f2-4fdd-a598-ea6656ed13a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:09.510161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1002/2036] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:30:09 2022] DELETE /identity/v3/projects/ee49c49b426648b9a3b5fae15f24bcea => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:09.513608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648bc2c-7e69-4516-ae10-1b760dde5aea None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:09.513608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648bc2c-7e69-4516-ae10-1b760dde5aea None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:09.513608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648bc2c-7e69-4516-ae10-1b760dde5aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:09.534014 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-9648bc2c-7e69-4516-ae10-1b760dde5aea None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:09.535234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9648bc2c-7e69-4516-ae10-1b760dde5aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:09.553176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:09.596184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1035/2037] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:09.604088 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f2d3-5db0-4eee-89f0-cead1c58664a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:09.604378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f2d3-5db0-4eee-89f0-cead1c58664a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:09.604656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f2d3-5db0-4eee-89f0-cead1c58664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:09.624292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0e77f2d3-5db0-4eee-89f0-cead1c58664a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:09.625530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0e77f2d3-5db0-4eee-89f0-cead1c58664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:09.634183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1003/2038] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:09.638509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648bc2c-7e69-4516-ae10-1b760dde5aea None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:09.649497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648bc2c-7e69-4516-ae10-1b760dde5aea None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:09.654167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7758f23-a3cf-44e7-b7a2-f8772a8903ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:09.654784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7758f23-a3cf-44e7-b7a2-f8772a8903ef admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:09.655072 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7758f23-a3cf-44e7-b7a2-f8772a8903ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:09.655404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7758f23-a3cf-44e7-b7a2-f8772a8903ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:09.656114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7758f23-a3cf-44e7-b7a2-f8772a8903ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:30:09.656426 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7758f23-a3cf-44e7-b7a2-f8772a8903ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:09.658080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7758f23-a3cf-44e7-b7a2-f8772a8903ef admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:09.669889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1036/2039] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:30:09.674300 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77f2d3-5db0-4eee-89f0-cead1c58664a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:09.683634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77f2d3-5db0-4eee-89f0-cead1c58664a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:09.688287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899b047-6ba2-483f-a627-a94deedc59c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:09.688895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899b047-6ba2-483f-a627-a94deedc59c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:09.689181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899b047-6ba2-483f-a627-a94deedc59c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:09.689500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899b047-6ba2-483f-a627-a94deedc59c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:09.690178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899b047-6ba2-483f-a627-a94deedc59c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:09.692101 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899b047-6ba2-483f-a627-a94deedc59c8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:09.774072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1004/2040] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:30:09 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:09.778267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7758f23-a3cf-44e7-b7a2-f8772a8903ef admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:09.788414 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7758f23-a3cf-44e7-b7a2-f8772a8903ef admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:09.854291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a862f1-ebd8-4d3a-834b-9fd68dd4a53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:09.854925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a862f1-ebd8-4d3a-834b-9fd68dd4a53c admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:09.855217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a862f1-ebd8-4d3a-834b-9fd68dd4a53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:09.855502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a862f1-ebd8-4d3a-834b-9fd68dd4a53c admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:09.856190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a862f1-ebd8-4d3a-834b-9fd68dd4a53c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:09.858194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a862f1-ebd8-4d3a-834b-9fd68dd4a53c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:09.978862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1037/2041] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:30:09 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:09.982716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899b047-6ba2-483f-a627-a94deedc59c8 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:09.993505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899b047-6ba2-483f-a627-a94deedc59c8 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:09.998129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3b0a0-a4c1-4621-b07e-fa570d634f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:09.998748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b0a0-a4c1-4621-b07e-fa570d634f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:09.999037 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b0a0-a4c1-4621-b07e-fa570d634f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:09.999379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b0a0-a4c1-4621-b07e-fa570d634f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:10.000067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3b0a0-a4c1-4621-b07e-fa570d634f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:10.001712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3b0a0-a4c1-4621-b07e-fa570d634f80 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:10.012084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1005/2042] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:10.016049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a862f1-ebd8-4d3a-834b-9fd68dd4a53c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:10.027168 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a862f1-ebd8-4d3a-834b-9fd68dd4a53c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:10.032290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b03e01-f911-45b6-b751-989b9e35b163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:10.032920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03e01-f911-45b6-b751-989b9e35b163 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:10.033207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03e01-f911-45b6-b751-989b9e35b163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:10.033512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03e01-f911-45b6-b751-989b9e35b163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:10.034221 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b03e01-f911-45b6-b751-989b9e35b163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:10.035847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b03e01-f911-45b6-b751-989b9e35b163 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:10.047619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1038/2043] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:10.051824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3b0a0-a4c1-4621-b07e-fa570d634f80 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:10.062006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3b0a0-a4c1-4621-b07e-fa570d634f80 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:10.066598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe369bfe-24bc-4167-b30e-1f75754b1ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:10.067265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe369bfe-24bc-4167-b30e-1f75754b1ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:10.067551 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe369bfe-24bc-4167-b30e-1f75754b1ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:10.067828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe369bfe-24bc-4167-b30e-1f75754b1ac2 admin admin] PATH_INFO: `/v3/projects/43637c0d10a14c7cb80886bfe03273c0/users/b1821b1a76274b27bcc1c1f4333e9480/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:10.082514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe369bfe-24bc-4167-b30e-1f75754b1ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43637c0d10a14c7cb80886bfe03273c0, user_id=b1821b1a76274b27bcc1c1f4333e9480, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:10.085177 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe369bfe-24bc-4167-b30e-1f75754b1ac2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:10.150339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1006/2044] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:10 2022] PUT /identity/v3/projects/43637c0d10a14c7cb80886bfe03273c0/users/b1821b1a76274b27bcc1c1f4333e9480/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:10.160858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c713b3-532a-4fd2-acd3-7a28135201cb None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:10.161143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c713b3-532a-4fd2-acd3-7a28135201cb None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:10.161458 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c713b3-532a-4fd2-acd3-7a28135201cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:10.191603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b5c713b3-532a-4fd2-acd3-7a28135201cb None None] MFA Rules not processed for user `b1821b1a76274b27bcc1c1f4333e9480`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:10.192738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b5c713b3-532a-4fd2-acd3-7a28135201cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:10.263734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1039/2045] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:10.281874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe369bfe-24bc-4167-b30e-1f75754b1ac2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:10.283707 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fe369bfe-24bc-4167-b30e-1f75754b1ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:10.307825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:10.365632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe369bfe-24bc-4167-b30e-1f75754b1ac2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:10.369720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92c9bb-9370-4b6b-b0ad-07e3723df118 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4fe04b11ec412e8f4eb67c6e54b8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:10.370356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92c9bb-9370-4b6b-b0ad-07e3723df118 service glance] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:10.370695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92c9bb-9370-4b6b-b0ad-07e3723df118 service glance] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:10.370980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92c9bb-9370-4b6b-b0ad-07e3723df118 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:10.379919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92c9bb-9370-4b6b-b0ad-07e3723df118 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:10.381396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92c9bb-9370-4b6b-b0ad-07e3723df118 service glance] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:10.391096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1007/2046] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:13.999043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c713b3-532a-4fd2-acd3-7a28135201cb None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:14.000439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b5c713b3-532a-4fd2-acd3-7a28135201cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:14.028998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:14.076062 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c713b3-532a-4fd2-acd3-7a28135201cb None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:14.084489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:14.085459 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:14.085912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:14.086360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] PATH_INFO: `/v3/users/189a5115ddf94081ad010a1a01df10ed` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:14.088602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=189a5115ddf94081ad010a1a01df10ed)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:14.091952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:14.344606 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '189a5115ddf94081ad010a1a01df10ed'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:14.363262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '189a5115ddf94081ad010a1a01df10ed'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:14.365973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '189a5115ddf94081ad010a1a01df10ed'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:14.417379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '189a5115ddf94081ad010a1a01df10ed'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:14.469387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '189a5115ddf94081ad010a1a01df10ed'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:14.481343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1040/2047] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:30:13 2022] DELETE /identity/v3/users/189a5115ddf94081ad010a1a01df10ed => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:14.484429 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714047c6-2a9e-4b89-bee5-4a8f9eef2e3d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:14.484923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714047c6-2a9e-4b89-bee5-4a8f9eef2e3d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:14.484923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714047c6-2a9e-4b89-bee5-4a8f9eef2e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:14.506466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-714047c6-2a9e-4b89-bee5-4a8f9eef2e3d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:14.508102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-714047c6-2a9e-4b89-bee5-4a8f9eef2e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:14.585739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1008/2048] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:14.597178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:14.598694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:14.666553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da4371-1128-4bdf-8889-d5ee8b9ba4b5 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:14.670350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba2df8-7fca-4b16-919e-6bd1d6b35f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:14.670976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2df8-7fca-4b16-919e-6bd1d6b35f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:14.671291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2df8-7fca-4b16-919e-6bd1d6b35f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:14.671569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2df8-7fca-4b16-919e-6bd1d6b35f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:14.688040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba2df8-7fca-4b16-919e-6bd1d6b35f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:14.689923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba2df8-7fca-4b16-919e-6bd1d6b35f64 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:14.700674 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1041/2049] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:15.099812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714047c6-2a9e-4b89-bee5-4a8f9eef2e3d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:15.101895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-714047c6-2a9e-4b89-bee5-4a8f9eef2e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:15.117064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714047c6-2a9e-4b89-bee5-4a8f9eef2e3d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:15.122052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:15.122759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:15.123048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:15.123329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] PATH_INFO: `/v3/projects/310e955c99624dc88a38254e9a36ca20` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:15.124525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=310e955c99624dc88a38254e9a36ca20)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:15.126499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:15.219236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310e955c99624dc88a38254e9a36ca20', 'request_id': 'req-e4578e27-8398-4bf2-ac49-5a16afe240e9'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:15.224754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310e955c99624dc88a38254e9a36ca20', 'request_id': 'req-e4578e27-8398-4bf2-ac49-5a16afe240e9'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:15.231979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310e955c99624dc88a38254e9a36ca20', 'request_id': 'req-e4578e27-8398-4bf2-ac49-5a16afe240e9'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:15.303151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] The token cache is being invalidate because project 310e955c99624dc88a38254e9a36ca20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:30:15.303561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:15.304413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e4578e27-8398-4bf2-ac49-5a16afe240e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:15.306017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1009/2050] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:30:15 2022] DELETE /identity/v3/projects/310e955c99624dc88a38254e9a36ca20 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:15.310858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e631d00-d108-4d7d-8d2f-848af3e16d4d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:15.311344 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e631d00-d108-4d7d-8d2f-848af3e16d4d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:15.311432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e631d00-d108-4d7d-8d2f-848af3e16d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:15.331145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-1e631d00-d108-4d7d-8d2f-848af3e16d4d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:15.332373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1e631d00-d108-4d7d-8d2f-848af3e16d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:15.372440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:15.399633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1042/2051] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:15.408683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b0afb-be49-47f1-803b-2d8eebd284a4 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:15.408968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b0afb-be49-47f1-803b-2d8eebd284a4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:15.409254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b0afb-be49-47f1-803b-2d8eebd284a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:15.433153 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-165b0afb-be49-47f1-803b-2d8eebd284a4 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:15.434405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-165b0afb-be49-47f1-803b-2d8eebd284a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:15.443211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1010/2052] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:15.448522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e631d00-d108-4d7d-8d2f-848af3e16d4d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:15.462938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e631d00-d108-4d7d-8d2f-848af3e16d4d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:15.468406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca26103-9f90-4a79-954d-885fc667cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:15.473341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca26103-9f90-4a79-954d-885fc667cfce admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:15.473341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca26103-9f90-4a79-954d-885fc667cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:15.473341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca26103-9f90-4a79-954d-885fc667cfce admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:15.473341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca26103-9f90-4a79-954d-885fc667cfce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:30:15.473341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca26103-9f90-4a79-954d-885fc667cfce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:15.473341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca26103-9f90-4a79-954d-885fc667cfce admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:15.484821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1043/2053] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:30:15.499636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b0afb-be49-47f1-803b-2d8eebd284a4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:15.527250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b0afb-be49-47f1-803b-2d8eebd284a4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:15.532302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afd044-4810-4793-9480-0f88efab6a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:15.532942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afd044-4810-4793-9480-0f88efab6a0f admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:15.533230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afd044-4810-4793-9480-0f88efab6a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:15.533644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afd044-4810-4793-9480-0f88efab6a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:15.534498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afd044-4810-4793-9480-0f88efab6a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:15.536345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afd044-4810-4793-9480-0f88efab6a0f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:15.618099 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:15.627612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1011/2054] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:30:15 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:15.634516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca26103-9f90-4a79-954d-885fc667cfce admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:15.646466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca26103-9f90-4a79-954d-885fc667cfce admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:15.730445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3f30e-b1fe-4e2d-acec-f4ee427ca3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:15.731078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3f30e-b1fe-4e2d-acec-f4ee427ca3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:15.731353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3f30e-b1fe-4e2d-acec-f4ee427ca3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:15.731818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3f30e-b1fe-4e2d-acec-f4ee427ca3f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:15.732581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3f30e-b1fe-4e2d-acec-f4ee427ca3f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:15.734941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3f30e-b1fe-4e2d-acec-f4ee427ca3f1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:15.845365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1044/2055] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:30:15 2022] POST /identity/v3/users => generated 287 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:15.849863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afd044-4810-4793-9480-0f88efab6a0f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:15.863591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afd044-4810-4793-9480-0f88efab6a0f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:15.870942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fedda2-448d-4f8b-8f25-798c499aeaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:15.871585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fedda2-448d-4f8b-8f25-798c499aeaf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:15.871872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fedda2-448d-4f8b-8f25-798c499aeaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:15.872156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fedda2-448d-4f8b-8f25-798c499aeaf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:15.872873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fedda2-448d-4f8b-8f25-798c499aeaf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:15.874742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fedda2-448d-4f8b-8f25-798c499aeaf0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:15.893370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1012/2056] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:15.896965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3f30e-b1fe-4e2d-acec-f4ee427ca3f1 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:15.912536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3f30e-b1fe-4e2d-acec-f4ee427ca3f1 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:15.918054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afb2f4-3f1b-43a5-a997-80ade323eccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:15.918871 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afb2f4-3f1b-43a5-a997-80ade323eccd admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:15.919659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afb2f4-3f1b-43a5-a997-80ade323eccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:15.919947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afb2f4-3f1b-43a5-a997-80ade323eccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:15.921418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afb2f4-3f1b-43a5-a997-80ade323eccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:15.923258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afb2f4-3f1b-43a5-a997-80ade323eccd admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:15.939068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1045/2057] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:15.943518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fedda2-448d-4f8b-8f25-798c499aeaf0 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:15.957213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fedda2-448d-4f8b-8f25-798c499aeaf0 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:15.963478 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4d6db-5427-4859-8368-d6a01c9b5c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:15.964154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4d6db-5427-4859-8368-d6a01c9b5c56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:15.964441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4d6db-5427-4859-8368-d6a01c9b5c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:15.964849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4d6db-5427-4859-8368-d6a01c9b5c56 admin admin] PATH_INFO: `/v3/projects/08e27723b1224589a896eab04740dffc/users/85d2763ce8a64f128f28ae36a50a8890/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:15.982984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4d6db-5427-4859-8368-d6a01c9b5c56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e27723b1224589a896eab04740dffc, user_id=85d2763ce8a64f128f28ae36a50a8890, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:15.985980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4d6db-5427-4859-8368-d6a01c9b5c56 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:16.065273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1013/2058] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:15 2022] PUT /identity/v3/projects/08e27723b1224589a896eab04740dffc/users/85d2763ce8a64f128f28ae36a50a8890/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:16.076327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f2aaa-ccfc-43b6-8d42-80e49815c499 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:16.076954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f2aaa-ccfc-43b6-8d42-80e49815c499 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:16.076954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f2aaa-ccfc-43b6-8d42-80e49815c499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:16.109814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f66f2aaa-ccfc-43b6-8d42-80e49815c499 None None] MFA Rules not processed for user `85d2763ce8a64f128f28ae36a50a8890`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:16.111045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f66f2aaa-ccfc-43b6-8d42-80e49815c499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:16.190346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1046/2059] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:16.207481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4d6db-5427-4859-8368-d6a01c9b5c56 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:16.209396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d7a4d6db-5427-4859-8368-d6a01c9b5c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:16.282590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4d6db-5427-4859-8368-d6a01c9b5c56 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:16.286427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c248d5f-f639-41ea-9920-2b1018091fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:16.287054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248d5f-f639-41ea-9920-2b1018091fbd service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:16.287343 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248d5f-f639-41ea-9920-2b1018091fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:16.287623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248d5f-f639-41ea-9920-2b1018091fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:16.296674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c248d5f-f639-41ea-9920-2b1018091fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:16.298135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c248d5f-f639-41ea-9920-2b1018091fbd service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:16.307737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1014/2060] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:16.994940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f2aaa-ccfc-43b6-8d42-80e49815c499 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:16.996764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f66f2aaa-ccfc-43b6-8d42-80e49815c499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:17.052359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:17.078300 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f2aaa-ccfc-43b6-8d42-80e49815c499 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:17.083154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:17.083803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:17.084083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:17.084798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] PATH_INFO: `/v3/users/ff512722c4134d368652f87fcdb9ce21` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:17.085982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff512722c4134d368652f87fcdb9ce21)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:17.087937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:17.306584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff512722c4134d368652f87fcdb9ce21'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:17.321699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff512722c4134d368652f87fcdb9ce21'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:17.322478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff512722c4134d368652f87fcdb9ce21'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:17.330189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff512722c4134d368652f87fcdb9ce21'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:17.480763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff512722c4134d368652f87fcdb9ce21'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:17.488095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1047/2061] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:30:16 2022] DELETE /identity/v3/users/ff512722c4134d368652f87fcdb9ce21 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:17.493556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b194a-3136-4a1a-ae44-246dcdbb4e18 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:17.494176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b194a-3136-4a1a-ae44-246dcdbb4e18 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:17.494176 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b194a-3136-4a1a-ae44-246dcdbb4e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:17.515735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1e2b194a-3136-4a1a-ae44-246dcdbb4e18 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:17.516981 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1e2b194a-3136-4a1a-ae44-246dcdbb4e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:17.581394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1015/2062] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:17.592058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:17.593568 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:17.659058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac2ab3-0b26-4144-b578-3f614e1e1b5d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:17.662739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2658ea-9466-4861-aa5e-24627650bd26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:17.663970 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2658ea-9466-4861-aa5e-24627650bd26 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:17.664258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2658ea-9466-4861-aa5e-24627650bd26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:17.664628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2658ea-9466-4861-aa5e-24627650bd26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:17.674333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2658ea-9466-4861-aa5e-24627650bd26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:17.675749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2658ea-9466-4861-aa5e-24627650bd26 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:17.685807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1048/2063] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:18.128377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b194a-3136-4a1a-ae44-246dcdbb4e18 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:18.130955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1e2b194a-3136-4a1a-ae44-246dcdbb4e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:18.151148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b194a-3136-4a1a-ae44-246dcdbb4e18 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:18.155946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:18.156613 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:18.156904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:18.157183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] PATH_INFO: `/v3/projects/7482210a43074a4c8a0db84e7ba3b704` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:18.158447 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7482210a43074a4c8a0db84e7ba3b704)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:18.160570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:18.311277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7482210a43074a4c8a0db84e7ba3b704', 'request_id': 'req-bdc49043-e4fd-412f-b5aa-fb076c2842f4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:18.318058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7482210a43074a4c8a0db84e7ba3b704', 'request_id': 'req-bdc49043-e4fd-412f-b5aa-fb076c2842f4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:18.320157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:18.324893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7482210a43074a4c8a0db84e7ba3b704', 'request_id': 'req-bdc49043-e4fd-412f-b5aa-fb076c2842f4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:18.429098 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] The token cache is being invalidate because project 7482210a43074a4c8a0db84e7ba3b704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:30:18.429426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:18.430061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:18.431600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1016/2064] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:30:18 2022] DELETE /identity/v3/projects/7482210a43074a4c8a0db84e7ba3b704 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:18.435781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2658ea-9466-4861-aa5e-24627650bd26 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:18.448016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2658ea-9466-4861-aa5e-24627650bd26 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:18.500390 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:18.506285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:18.507056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:18.507437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:18.507723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] PATH_INFO: `/v3/users/9a438defe4ef49bdb32edd794340442b` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:18.508978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a438defe4ef49bdb32edd794340442b)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:18.510933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:18.637480 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a438defe4ef49bdb32edd794340442b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:18.650083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a438defe4ef49bdb32edd794340442b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:18.650776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a438defe4ef49bdb32edd794340442b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:18.657714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a438defe4ef49bdb32edd794340442b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:18.708844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-4ad5ac0b-b613-4d77-9585-538e3a326583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a438defe4ef49bdb32edd794340442b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:18.716328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1049/2065] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:30:18 2022] DELETE /identity/v3/users/9a438defe4ef49bdb32edd794340442b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:19.141858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:19.144342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:19.177538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc49043-e4fd-412f-b5aa-fb076c2842f4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:19.226302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:19.226947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:19.227264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:19.227542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] PATH_INFO: `/v3/projects/3a94a6fa27d547cd9144399d736e2eda` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:19.228767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a94a6fa27d547cd9144399d736e2eda)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:19.230706 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:19.325813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a94a6fa27d547cd9144399d736e2eda', 'request_id': 'req-d4704583-072a-4733-a149-8a0c6af58fac'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:19.331714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a94a6fa27d547cd9144399d736e2eda', 'request_id': 'req-d4704583-072a-4733-a149-8a0c6af58fac'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:19.338784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a94a6fa27d547cd9144399d736e2eda', 'request_id': 'req-d4704583-072a-4733-a149-8a0c6af58fac'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:19.417813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] The token cache is being invalidate because project 3a94a6fa27d547cd9144399d736e2eda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:30:19.418006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:19.418632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d4704583-072a-4733-a149-8a0c6af58fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:19.420188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1017/2066] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:30:19 2022] DELETE /identity/v3/projects/3a94a6fa27d547cd9144399d736e2eda => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:19.425292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101de28-8a6b-495f-b29d-4b25d427d3a1 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:19.425608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101de28-8a6b-495f-b29d-4b25d427d3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:19.425892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101de28-8a6b-495f-b29d-4b25d427d3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:19.446461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-7101de28-8a6b-495f-b29d-4b25d427d3a1 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:19.447726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7101de28-8a6b-495f-b29d-4b25d427d3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:19.509638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1050/2067] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:19.517273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9415af-3037-4a4e-b528-87cb7cce20a7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:19.517654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9415af-3037-4a4e-b528-87cb7cce20a7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:19.517951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9415af-3037-4a4e-b528-87cb7cce20a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:19.537891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6a9415af-3037-4a4e-b528-87cb7cce20a7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:19.539607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6a9415af-3037-4a4e-b528-87cb7cce20a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:19.547443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1018/2068] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:19.551693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101de28-8a6b-495f-b29d-4b25d427d3a1 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:19.562444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101de28-8a6b-495f-b29d-4b25d427d3a1 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:19.567326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114f47c-5e2b-4622-b89e-6b93df6df7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:19.567931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f47c-5e2b-4622-b89e-6b93df6df7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:19.568220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f47c-5e2b-4622-b89e-6b93df6df7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:19.568502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f47c-5e2b-4622-b89e-6b93df6df7f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:19.569240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114f47c-5e2b-4622-b89e-6b93df6df7f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:30:19.569742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114f47c-5e2b-4622-b89e-6b93df6df7f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:19.571374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114f47c-5e2b-4622-b89e-6b93df6df7f4 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:19.582988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1051/2069] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:30:19.587306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9415af-3037-4a4e-b528-87cb7cce20a7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:19.597005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9415af-3037-4a4e-b528-87cb7cce20a7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:19.602422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36edb71-c7e3-4127-85f1-9f82e4229531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:19.603031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36edb71-c7e3-4127-85f1-9f82e4229531 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:19.603316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36edb71-c7e3-4127-85f1-9f82e4229531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:19.603594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36edb71-c7e3-4127-85f1-9f82e4229531 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:19.604266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36edb71-c7e3-4127-85f1-9f82e4229531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:19.606223 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36edb71-c7e3-4127-85f1-9f82e4229531 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:19.733655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1019/2070] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:30:19 2022] POST /identity/v3/projects => generated 391 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:19.737945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114f47c-5e2b-4622-b89e-6b93df6df7f4 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:19.748894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114f47c-5e2b-4622-b89e-6b93df6df7f4 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:19.807452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d71d98-ff6a-4285-b81d-d07e0a769118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:19.808098 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d71d98-ff6a-4285-b81d-d07e0a769118 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:19.808383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d71d98-ff6a-4285-b81d-d07e0a769118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:19.808658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d71d98-ff6a-4285-b81d-d07e0a769118 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:19.809392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d71d98-ff6a-4285-b81d-d07e0a769118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:19.811362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d71d98-ff6a-4285-b81d-d07e0a769118 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:19.822539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:19.927085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1052/2071] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:30:19 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:19.931987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36edb71-c7e3-4127-85f1-9f82e4229531 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:19.942873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36edb71-c7e3-4127-85f1-9f82e4229531 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:19.948736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10e4e-94ec-4c9a-ab35-824b8092a04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:19.949391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10e4e-94ec-4c9a-ab35-824b8092a04d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:19.949700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10e4e-94ec-4c9a-ab35-824b8092a04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:19.949976 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10e4e-94ec-4c9a-ab35-824b8092a04d admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:19.950676 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10e4e-94ec-4c9a-ab35-824b8092a04d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:19.952341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10e4e-94ec-4c9a-ab35-824b8092a04d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:19.962604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1020/2072] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:19.967011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d71d98-ff6a-4285-b81d-d07e0a769118 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:19.977080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d71d98-ff6a-4285-b81d-d07e0a769118 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:19.981741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e93e83-1c34-432e-a1ca-be2d3caba810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:19.982347 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e93e83-1c34-432e-a1ca-be2d3caba810 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:19.982633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e93e83-1c34-432e-a1ca-be2d3caba810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:19.982912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e93e83-1c34-432e-a1ca-be2d3caba810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:19.983608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e93e83-1c34-432e-a1ca-be2d3caba810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:19.985243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e93e83-1c34-432e-a1ca-be2d3caba810 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:19.997013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1053/2073] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:20.001111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10e4e-94ec-4c9a-ab35-824b8092a04d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:20.011816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10e4e-94ec-4c9a-ab35-824b8092a04d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:20.016489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138070e-48f4-4778-bad5-70f61263330e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:20.017152 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138070e-48f4-4778-bad5-70f61263330e admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:20.017482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138070e-48f4-4778-bad5-70f61263330e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:20.017766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138070e-48f4-4778-bad5-70f61263330e admin admin] PATH_INFO: `/v3/projects/8d9c4561ad9b4bc987be6d3a9afac1a8/users/7d511f18f34148ed9a83c07e4f687486/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:20.032844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138070e-48f4-4778-bad5-70f61263330e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9c4561ad9b4bc987be6d3a9afac1a8, user_id=7d511f18f34148ed9a83c07e4f687486, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:20.035571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138070e-48f4-4778-bad5-70f61263330e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:20.104617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1021/2074] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:20 2022] PUT /identity/v3/projects/8d9c4561ad9b4bc987be6d3a9afac1a8/users/7d511f18f34148ed9a83c07e4f687486/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:20.180059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66055b83-43ff-47fe-b108-7c8e0bc0ed56 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:20.180484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66055b83-43ff-47fe-b108-7c8e0bc0ed56 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:20.180689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66055b83-43ff-47fe-b108-7c8e0bc0ed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:20.210946 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-66055b83-43ff-47fe-b108-7c8e0bc0ed56 None None] MFA Rules not processed for user `7d511f18f34148ed9a83c07e4f687486`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:20.212169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-66055b83-43ff-47fe-b108-7c8e0bc0ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:20.293930 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1054/2075] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:20.302120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138070e-48f4-4778-bad5-70f61263330e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:20.322929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:20.381110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138070e-48f4-4778-bad5-70f61263330e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:20.386309 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f299404-4b49-489b-a3e0-b1cb3bfc3eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:20.386935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f299404-4b49-489b-a3e0-b1cb3bfc3eff admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:20.387220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f299404-4b49-489b-a3e0-b1cb3bfc3eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:20.387524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f299404-4b49-489b-a3e0-b1cb3bfc3eff admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:20.388167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f299404-4b49-489b-a3e0-b1cb3bfc3eff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:20.390229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f299404-4b49-489b-a3e0-b1cb3bfc3eff admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:20.471771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1022/2076] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:30:20 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:20.474853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66055b83-43ff-47fe-b108-7c8e0bc0ed56 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:20.486487 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66055b83-43ff-47fe-b108-7c8e0bc0ed56 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:20.547773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3718b71-c0bd-418e-a9a1-0f689516e233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:20.548342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3718b71-c0bd-418e-a9a1-0f689516e233 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:20.548585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3718b71-c0bd-418e-a9a1-0f689516e233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:20.548864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3718b71-c0bd-418e-a9a1-0f689516e233 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:20.549583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3718b71-c0bd-418e-a9a1-0f689516e233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:20.551517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3718b71-c0bd-418e-a9a1-0f689516e233 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:20.623741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1055/2077] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:30:20 2022] POST /identity/v3/users => generated 302 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:20.627878 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f299404-4b49-489b-a3e0-b1cb3bfc3eff admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:20.638960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f299404-4b49-489b-a3e0-b1cb3bfc3eff admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:20.643629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d973b15-bd3d-4cc4-b119-e45fe21839ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:20.644362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d973b15-bd3d-4cc4-b119-e45fe21839ec admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:20.644654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d973b15-bd3d-4cc4-b119-e45fe21839ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:20.644940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d973b15-bd3d-4cc4-b119-e45fe21839ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:20.645750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d973b15-bd3d-4cc4-b119-e45fe21839ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:20.647916 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d973b15-bd3d-4cc4-b119-e45fe21839ec admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:20.659452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1023/2078] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:20.664046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3718b71-c0bd-418e-a9a1-0f689516e233 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:20.674109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3718b71-c0bd-418e-a9a1-0f689516e233 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:20.679364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73bd33-efcf-4644-a16c-cedc47a1abb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:20.680061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73bd33-efcf-4644-a16c-cedc47a1abb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:20.680344 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73bd33-efcf-4644-a16c-cedc47a1abb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:20.680628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73bd33-efcf-4644-a16c-cedc47a1abb6 admin admin] PATH_INFO: `/v3/projects/2fdf356130ee4d5791dd4be7d909f784/users/30578743c059436380295e5d11ef02da/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:20.696524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73bd33-efcf-4644-a16c-cedc47a1abb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fdf356130ee4d5791dd4be7d909f784, user_id=30578743c059436380295e5d11ef02da, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:20.699228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73bd33-efcf-4644-a16c-cedc47a1abb6 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:20.772644 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1056/2079] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:20 2022] PUT /identity/v3/projects/2fdf356130ee4d5791dd4be7d909f784/users/30578743c059436380295e5d11ef02da/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:20.777209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d973b15-bd3d-4cc4-b119-e45fe21839ec admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:20.788415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d973b15-bd3d-4cc4-b119-e45fe21839ec admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:20.853004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485f1a3-082c-4cdc-89fe-06dbf8e7e4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:20.853695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485f1a3-082c-4cdc-89fe-06dbf8e7e4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:20.853973 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485f1a3-082c-4cdc-89fe-06dbf8e7e4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:20.854251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485f1a3-082c-4cdc-89fe-06dbf8e7e4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:20.855031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485f1a3-082c-4cdc-89fe-06dbf8e7e4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:20.856606 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485f1a3-082c-4cdc-89fe-06dbf8e7e4c2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:20.867361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1024/2080] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:20.871579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73bd33-efcf-4644-a16c-cedc47a1abb6 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:20.882160 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73bd33-efcf-4644-a16c-cedc47a1abb6 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:20.886879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba8bbb-5063-4013-9960-fa2e0a86cb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:20.887551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8bbb-5063-4013-9960-fa2e0a86cb0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:20.887872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8bbb-5063-4013-9960-fa2e0a86cb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:20.888155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8bbb-5063-4013-9960-fa2e0a86cb0e admin admin] PATH_INFO: `/v3/projects/2fdf356130ee4d5791dd4be7d909f784/users/30578743c059436380295e5d11ef02da/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:20.890161 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba8bbb-5063-4013-9960-fa2e0a86cb0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fdf356130ee4d5791dd4be7d909f784, user_id=30578743c059436380295e5d11ef02da, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:20.892811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba8bbb-5063-4013-9960-fa2e0a86cb0e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:20.903343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1057/2081] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:20 2022] PUT /identity/v3/projects/2fdf356130ee4d5791dd4be7d909f784/users/30578743c059436380295e5d11ef02da/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:20.914059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986bca7-8796-4afb-a52a-d86655372fc5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:20.914467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986bca7-8796-4afb-a52a-d86655372fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:20.914762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986bca7-8796-4afb-a52a-d86655372fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:20.944744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d986bca7-8796-4afb-a52a-d86655372fc5 None None] MFA Rules not processed for user `30578743c059436380295e5d11ef02da`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:20.946050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d986bca7-8796-4afb-a52a-d86655372fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:21.013843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1025/2082] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:21.024536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba8bbb-5063-4013-9960-fa2e0a86cb0e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:21.026452 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0fba8bbb-5063-4013-9960-fa2e0a86cb0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:21.056280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:21.085001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba8bbb-5063-4013-9960-fa2e0a86cb0e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:21.089054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0eef22-56a2-46f9-a740-8e60e88dad70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:21.089708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0eef22-56a2-46f9-a740-8e60e88dad70 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:21.089998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0eef22-56a2-46f9-a740-8e60e88dad70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:21.090283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0eef22-56a2-46f9-a740-8e60e88dad70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:21.099793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0eef22-56a2-46f9-a740-8e60e88dad70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:21.105055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0eef22-56a2-46f9-a740-8e60e88dad70 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:21.111821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1058/2083] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:21.158966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986bca7-8796-4afb-a52a-d86655372fc5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:21.160367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d986bca7-8796-4afb-a52a-d86655372fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:21.173201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986bca7-8796-4afb-a52a-d86655372fc5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:21.177284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2003a-a386-4e46-ac94-dcb37d6ff185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:21.177999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2003a-a386-4e46-ac94-dcb37d6ff185 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:21.178291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2003a-a386-4e46-ac94-dcb37d6ff185 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:21.178728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2003a-a386-4e46-ac94-dcb37d6ff185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:21.188095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:21.248742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2003a-a386-4e46-ac94-dcb37d6ff185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:21.250552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2003a-a386-4e46-ac94-dcb37d6ff185 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:21.260453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1026/2084] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:21.294092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca768a1-2403-4a76-9051-823e9d1ff24c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:21.295543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca768a1-2403-4a76-9051-823e9d1ff24c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:21.295543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca768a1-2403-4a76-9051-823e9d1ff24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:21.321256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-cca768a1-2403-4a76-9051-823e9d1ff24c None None] MFA Rules not processed for user `85d2763ce8a64f128f28ae36a50a8890`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:21.322724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-cca768a1-2403-4a76-9051-823e9d1ff24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:21.415666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1059/2085] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:21.908174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201a79f-e9f6-4380-acbe-3cbbc6bc82d8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:21.908742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201a79f-e9f6-4380-acbe-3cbbc6bc82d8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:21.908742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201a79f-e9f6-4380-acbe-3cbbc6bc82d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:21.959392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d201a79f-e9f6-4380-acbe-3cbbc6bc82d8 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:21.962083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d201a79f-e9f6-4380-acbe-3cbbc6bc82d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:22.035032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1027/2086] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:30:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:22.089827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca768a1-2403-4a76-9051-823e9d1ff24c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:22.091384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-cca768a1-2403-4a76-9051-823e9d1ff24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:22.199721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca768a1-2403-4a76-9051-823e9d1ff24c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:22.206357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b802f2d-2fed-4959-93cd-45e07ba4ecf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:22.209719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b802f2d-2fed-4959-93cd-45e07ba4ecf9 service swift] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:22.209719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b802f2d-2fed-4959-93cd-45e07ba4ecf9 service swift] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:22.209719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b802f2d-2fed-4959-93cd-45e07ba4ecf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:22.217587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b802f2d-2fed-4959-93cd-45e07ba4ecf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:22.219395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b802f2d-2fed-4959-93cd-45e07ba4ecf9 service swift] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:22.228922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1060/2087] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:30:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:30:26.836365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201a79f-e9f6-4380-acbe-3cbbc6bc82d8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:26.987391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201a79f-e9f6-4380-acbe-3cbbc6bc82d8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:26.998506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:27.007698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:27.007698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:27.007698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] PATH_INFO: `/v3/users/7d511f18f34148ed9a83c07e4f687486` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:27.007698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d511f18f34148ed9a83c07e4f687486)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:27.017633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:27.329954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d511f18f34148ed9a83c07e4f687486'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:27.367181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:27.371763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d511f18f34148ed9a83c07e4f687486'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:27.392622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d511f18f34148ed9a83c07e4f687486'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:27.510557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d511f18f34148ed9a83c07e4f687486'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:27.517076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d511f18f34148ed9a83c07e4f687486'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:27.564861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1028/2088] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:30:26 2022] DELETE /identity/v3/users/7d511f18f34148ed9a83c07e4f687486 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:27.566712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735cca-500c-4cf6-8bd4-5a6c7d68b0cf None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:27.567251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735cca-500c-4cf6-8bd4-5a6c7d68b0cf None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:27.567337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735cca-500c-4cf6-8bd4-5a6c7d68b0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:27.600971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-ec735cca-500c-4cf6-8bd4-5a6c7d68b0cf None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:27.603552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ec735cca-500c-4cf6-8bd4-5a6c7d68b0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:27.826091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1061/2089] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:27.834301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:27.837812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:28.009668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad621f6-06b3-44cf-8d60-e7829553b423 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:28.014169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28d127-32b9-4f47-809a-40187cf268f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:28.014745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28d127-32b9-4f47-809a-40187cf268f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:28.015090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28d127-32b9-4f47-809a-40187cf268f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:28.015436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28d127-32b9-4f47-809a-40187cf268f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:28.033417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28d127-32b9-4f47-809a-40187cf268f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:28.034879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28d127-32b9-4f47-809a-40187cf268f9 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:28.070458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1029/2090] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:28.718643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735cca-500c-4cf6-8bd4-5a6c7d68b0cf None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:28.723338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ec735cca-500c-4cf6-8bd4-5a6c7d68b0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:28.777894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:28.790412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735cca-500c-4cf6-8bd4-5a6c7d68b0cf None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:28.808389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:28.809298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:28.809744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:28.810110 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] PATH_INFO: `/v3/projects/8d9c4561ad9b4bc987be6d3a9afac1a8` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:28.812333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9c4561ad9b4bc987be6d3a9afac1a8)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:28.819332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:28.972107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9c4561ad9b4bc987be6d3a9afac1a8', 'request_id': 'req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:28.983869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9c4561ad9b4bc987be6d3a9afac1a8', 'request_id': 'req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:29.104460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9c4561ad9b4bc987be6d3a9afac1a8', 'request_id': 'req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:29.122696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] The token cache is being invalidate because project 8d9c4561ad9b4bc987be6d3a9afac1a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:30:29.123920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:29.138655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:29.140886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1062/2091] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:30:28 2022] DELETE /identity/v3/projects/8d9c4561ad9b4bc987be6d3a9afac1a8 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:29.144768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28d127-32b9-4f47-809a-40187cf268f9 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:29.158378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28d127-32b9-4f47-809a-40187cf268f9 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:29.238978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:29.240043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:29.240752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:29.241193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] PATH_INFO: `/v3/users/30578743c059436380295e5d11ef02da` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:29.243648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30578743c059436380295e5d11ef02da)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:29.247376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:29.546918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30578743c059436380295e5d11ef02da'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:29.586695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30578743c059436380295e5d11ef02da'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:29.595882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30578743c059436380295e5d11ef02da'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:29.602172 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:29.657782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30578743c059436380295e5d11ef02da'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:29.658912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-97124c6c-14ec-4035-a1f8-7cdd26210fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30578743c059436380295e5d11ef02da'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:29.667173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1030/2092] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:30:29 2022] DELETE /identity/v3/users/30578743c059436380295e5d11ef02da => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:30.264178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:30.269968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:30.282195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e6c5b-fd43-4848-918d-d264e6ceefc3 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:30.371560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:30.371560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:30.371560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:30.373383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] PATH_INFO: `/v3/projects/2fdf356130ee4d5791dd4be7d909f784` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:30.376478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fdf356130ee4d5791dd4be7d909f784)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:30.379359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:30.531340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fdf356130ee4d5791dd4be7d909f784', 'request_id': 'req-84b3e0d3-710c-42ff-992a-fc98d36b7269'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:30.540358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fdf356130ee4d5791dd4be7d909f784', 'request_id': 'req-84b3e0d3-710c-42ff-992a-fc98d36b7269'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:30.610826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fdf356130ee4d5791dd4be7d909f784', 'request_id': 'req-84b3e0d3-710c-42ff-992a-fc98d36b7269'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:30.634967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] The token cache is being invalidate because project 2fdf356130ee4d5791dd4be7d909f784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:30:30.635419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:30.636719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-84b3e0d3-710c-42ff-992a-fc98d36b7269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:30.638874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1063/2093] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:30:30 2022] DELETE /identity/v3/projects/2fdf356130ee4d5791dd4be7d909f784 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:30.644250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89096337-6393-49ee-8147-c22895333333 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:30.644904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89096337-6393-49ee-8147-c22895333333 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:30.644997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89096337-6393-49ee-8147-c22895333333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:30.673492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-89096337-6393-49ee-8147-c22895333333 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:30.676016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-89096337-6393-49ee-8147-c22895333333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:30.821418 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1031/2094] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:30.836563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ac40f-354f-4375-8efd-90e10a937576 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:30.837737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ac40f-354f-4375-8efd-90e10a937576 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:30.837997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ac40f-354f-4375-8efd-90e10a937576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:30.869114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-a60ac40f-354f-4375-8efd-90e10a937576 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:30.870910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a60ac40f-354f-4375-8efd-90e10a937576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:30.883295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1064/2095] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:30.894970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89096337-6393-49ee-8147-c22895333333 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:30.931794 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89096337-6393-49ee-8147-c22895333333 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:30.939886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a91a38-90fd-4e50-92ae-3d071f7b447d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:30.940912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a91a38-90fd-4e50-92ae-3d071f7b447d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:30.941430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a91a38-90fd-4e50-92ae-3d071f7b447d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:30.941919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a91a38-90fd-4e50-92ae-3d071f7b447d admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:30.946161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a91a38-90fd-4e50-92ae-3d071f7b447d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:30:30.947184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a91a38-90fd-4e50-92ae-3d071f7b447d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:30.952701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a91a38-90fd-4e50-92ae-3d071f7b447d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:30.976421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1032/2096] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:30:30.981687 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ac40f-354f-4375-8efd-90e10a937576 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:31.016971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ac40f-354f-4375-8efd-90e10a937576 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:31.039516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f12df6-8f23-4cd5-bc33-dfaccf7dbcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:31.040175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f12df6-8f23-4cd5-bc33-dfaccf7dbcb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:31.041413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f12df6-8f23-4cd5-bc33-dfaccf7dbcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:31.041709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f12df6-8f23-4cd5-bc33-dfaccf7dbcb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:31.042419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f12df6-8f23-4cd5-bc33-dfaccf7dbcb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:31.044402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f12df6-8f23-4cd5-bc33-dfaccf7dbcb8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:31.062374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:31.209864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1065/2097] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:30:30 2022] POST /identity/v3/projects => generated 399 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:31.211484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a91a38-90fd-4e50-92ae-3d071f7b447d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:31.228371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a91a38-90fd-4e50-92ae-3d071f7b447d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:31.399549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a1c14-51c1-46df-87c4-6cbf89dc4000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:31.400186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a1c14-51c1-46df-87c4-6cbf89dc4000 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:31.400466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a1c14-51c1-46df-87c4-6cbf89dc4000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:31.400741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a1c14-51c1-46df-87c4-6cbf89dc4000 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:31.401447 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a1c14-51c1-46df-87c4-6cbf89dc4000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:31.403402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a1c14-51c1-46df-87c4-6cbf89dc4000 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:31.558268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1033/2098] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:30:31 2022] POST /identity/v3/users => generated 300 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:31.559873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f12df6-8f23-4cd5-bc33-dfaccf7dbcb8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:31.574880 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f12df6-8f23-4cd5-bc33-dfaccf7dbcb8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:31.579781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a993d9-0667-430c-bab0-04a011d017e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:31.580653 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a993d9-0667-430c-bab0-04a011d017e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:31.581062 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a993d9-0667-430c-bab0-04a011d017e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:31.581671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a993d9-0667-430c-bab0-04a011d017e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:31.582661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a993d9-0667-430c-bab0-04a011d017e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:31.594336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a993d9-0667-430c-bab0-04a011d017e9 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:31.600769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1066/2099] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:31.604657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a1c14-51c1-46df-87c4-6cbf89dc4000 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:31.620542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a1c14-51c1-46df-87c4-6cbf89dc4000 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:31.627681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fc8ff-495b-4e4b-a04b-89817086a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:31.628527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fc8ff-495b-4e4b-a04b-89817086a7db admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:31.628929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fc8ff-495b-4e4b-a04b-89817086a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:31.629461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fc8ff-495b-4e4b-a04b-89817086a7db admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:31.630349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fc8ff-495b-4e4b-a04b-89817086a7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:31.633150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fc8ff-495b-4e4b-a04b-89817086a7db admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:31.655231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1034/2100] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:30:31.669199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a993d9-0667-430c-bab0-04a011d017e9 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:31.683842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a993d9-0667-430c-bab0-04a011d017e9 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:31.691619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33eab4-9ded-458c-a709-501fbdaa4989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:31.692509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33eab4-9ded-458c-a709-501fbdaa4989 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:31.692806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33eab4-9ded-458c-a709-501fbdaa4989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:31.693154 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33eab4-9ded-458c-a709-501fbdaa4989 admin admin] PATH_INFO: `/v3/projects/5feed1ec709e49c48bb9be30d3451c3b/users/2837531809ef457997a780caf0e04540/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:31.759465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33eab4-9ded-458c-a709-501fbdaa4989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5feed1ec709e49c48bb9be30d3451c3b, user_id=2837531809ef457997a780caf0e04540, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:31.762327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33eab4-9ded-458c-a709-501fbdaa4989 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:31.862507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1067/2101] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:31 2022] PUT /identity/v3/projects/5feed1ec709e49c48bb9be30d3451c3b/users/2837531809ef457997a780caf0e04540/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:31.878755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3187-aa75-4437-99f2-735c3acf77e5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:31.879045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3187-aa75-4437-99f2-735c3acf77e5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:31.879379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c3187-aa75-4437-99f2-735c3acf77e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:31.913487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d43c3187-aa75-4437-99f2-735c3acf77e5 None None] MFA Rules not processed for user `2837531809ef457997a780caf0e04540`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:31.914782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d43c3187-aa75-4437-99f2-735c3acf77e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:31.986858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:32.037347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1035/2102] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:32.039765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33eab4-9ded-458c-a709-501fbdaa4989 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:32.127813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33eab4-9ded-458c-a709-501fbdaa4989 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:32.133860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee913e24-9eea-4ef9-b1ca-6d5e82b4c381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:32.135758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee913e24-9eea-4ef9-b1ca-6d5e82b4c381 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:32.135758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee913e24-9eea-4ef9-b1ca-6d5e82b4c381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:32.135758 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee913e24-9eea-4ef9-b1ca-6d5e82b4c381 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:32.136091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee913e24-9eea-4ef9-b1ca-6d5e82b4c381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:32.138054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee913e24-9eea-4ef9-b1ca-6d5e82b4c381 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:32.224542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1068/2103] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:30:32 2022] POST /identity/v3/projects => generated 401 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:32.228933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c3187-aa75-4437-99f2-735c3acf77e5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:32.243747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c3187-aa75-4437-99f2-735c3acf77e5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:32.327305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c9ae4-5b09-40c3-a345-13d74bee3f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:32.331501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c9ae4-5b09-40c3-a345-13d74bee3f65 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:32.331501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c9ae4-5b09-40c3-a345-13d74bee3f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:32.331501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c9ae4-5b09-40c3-a345-13d74bee3f65 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:32.331501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c9ae4-5b09-40c3-a345-13d74bee3f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:32.331994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c9ae4-5b09-40c3-a345-13d74bee3f65 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:32.499249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1036/2104] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:30:32 2022] POST /identity/v3/users => generated 307 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:32.502560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee913e24-9eea-4ef9-b1ca-6d5e82b4c381 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:32.513666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee913e24-9eea-4ef9-b1ca-6d5e82b4c381 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:32.518235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62918bf-13d4-43fc-a3a3-eda6c3698774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:32.518846 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62918bf-13d4-43fc-a3a3-eda6c3698774 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:32.519132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62918bf-13d4-43fc-a3a3-eda6c3698774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:32.519412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62918bf-13d4-43fc-a3a3-eda6c3698774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:32.520109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62918bf-13d4-43fc-a3a3-eda6c3698774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:32.521746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62918bf-13d4-43fc-a3a3-eda6c3698774 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:32.534492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1069/2105] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:32.538856 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c9ae4-5b09-40c3-a345-13d74bee3f65 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:32.550052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c9ae4-5b09-40c3-a345-13d74bee3f65 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:32.555326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b88b5-b18f-4641-856e-b35c58484d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:32.556496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b88b5-b18f-4641-856e-b35c58484d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:32.556496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b88b5-b18f-4641-856e-b35c58484d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:32.556741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b88b5-b18f-4641-856e-b35c58484d3e admin admin] PATH_INFO: `/v3/projects/43c9e6be6b3e4d28a6f06274a8c6c1dd/users/1cda45eb2a4b4ef5a8589b6fed5a0977/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:32.575119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b88b5-b18f-4641-856e-b35c58484d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c9e6be6b3e4d28a6f06274a8c6c1dd, user_id=1cda45eb2a4b4ef5a8589b6fed5a0977, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:32.578032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b88b5-b18f-4641-856e-b35c58484d3e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:32.655108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1037/2106] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:32 2022] PUT /identity/v3/projects/43c9e6be6b3e4d28a6f06274a8c6c1dd/users/1cda45eb2a4b4ef5a8589b6fed5a0977/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:30:32.661410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62918bf-13d4-43fc-a3a3-eda6c3698774 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:32.674734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62918bf-13d4-43fc-a3a3-eda6c3698774 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:32.755079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90cb55-da6d-4c15-a7e0-779b033879dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:32.756055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90cb55-da6d-4c15-a7e0-779b033879dc admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:32.756608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90cb55-da6d-4c15-a7e0-779b033879dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:32.757105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90cb55-da6d-4c15-a7e0-779b033879dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:32.758684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90cb55-da6d-4c15-a7e0-779b033879dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:32.760460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90cb55-da6d-4c15-a7e0-779b033879dc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:32.777986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1070/2107] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:32.782002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b88b5-b18f-4641-856e-b35c58484d3e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:32.794259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b88b5-b18f-4641-856e-b35c58484d3e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:32.800350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6eb8c-5b4b-41de-9d90-b3703fa7cd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:32.801019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6eb8c-5b4b-41de-9d90-b3703fa7cd94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:32.801565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6eb8c-5b4b-41de-9d90-b3703fa7cd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:32.802087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6eb8c-5b4b-41de-9d90-b3703fa7cd94 admin admin] PATH_INFO: `/v3/projects/43c9e6be6b3e4d28a6f06274a8c6c1dd/users/1cda45eb2a4b4ef5a8589b6fed5a0977/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:32.808509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6eb8c-5b4b-41de-9d90-b3703fa7cd94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c9e6be6b3e4d28a6f06274a8c6c1dd, user_id=1cda45eb2a4b4ef5a8589b6fed5a0977, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:32.821137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6eb8c-5b4b-41de-9d90-b3703fa7cd94 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:32.854013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1038/2108] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:32 2022] PUT /identity/v3/projects/43c9e6be6b3e4d28a6f06274a8c6c1dd/users/1cda45eb2a4b4ef5a8589b6fed5a0977/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:32.864430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a9a91-4500-46c0-a79e-e8375e22c435 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:32.865010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a9a91-4500-46c0-a79e-e8375e22c435 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:32.865360 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a9a91-4500-46c0-a79e-e8375e22c435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:32.908315 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-3e4a9a91-4500-46c0-a79e-e8375e22c435 None None] MFA Rules not processed for user `1cda45eb2a4b4ef5a8589b6fed5a0977`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:32.909630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3e4a9a91-4500-46c0-a79e-e8375e22c435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:32.922642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:33.003204 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1071/2109] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:33.014148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6eb8c-5b4b-41de-9d90-b3703fa7cd94 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:33.015941 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-57f6eb8c-5b4b-41de-9d90-b3703fa7cd94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:33.095566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6eb8c-5b4b-41de-9d90-b3703fa7cd94 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:33.100352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaaf91-27fa-4847-9382-3d8244a69a5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:33.100352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaaf91-27fa-4847-9382-3d8244a69a5b service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:33.100352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaaf91-27fa-4847-9382-3d8244a69a5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:33.100703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaaf91-27fa-4847-9382-3d8244a69a5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:33.131218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deaaf91-27fa-4847-9382-3d8244a69a5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:33.132699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deaaf91-27fa-4847-9382-3d8244a69a5b service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:33.143839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1039/2110] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:33.441466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a9a91-4500-46c0-a79e-e8375e22c435 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:33.452935 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a9a91-4500-46c0-a79e-e8375e22c435 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:33.457022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f342472-9a46-4d16-9ee3-a5b618e5bc69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:33.457712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342472-9a46-4d16-9ee3-a5b618e5bc69 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:33.458017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342472-9a46-4d16-9ee3-a5b618e5bc69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:33.458286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342472-9a46-4d16-9ee3-a5b618e5bc69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:33.618739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f342472-9a46-4d16-9ee3-a5b618e5bc69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:33.621069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f342472-9a46-4d16-9ee3-a5b618e5bc69 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:33.644004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1072/2111] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:36.813390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b4f11-d3dd-4813-b127-307cdbdc14b0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:36.814302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b4f11-d3dd-4813-b127-307cdbdc14b0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:36.814738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b4f11-d3dd-4813-b127-307cdbdc14b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:36.845202 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-105b4f11-d3dd-4813-b127-307cdbdc14b0 None None] MFA Rules not processed for user `86a5553819d949ffbc64b7a388942baf`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:36.846632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-105b4f11-d3dd-4813-b127-307cdbdc14b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:36.895539 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:36.911199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1040/2112] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:39.780898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f342472-9a46-4d16-9ee3-a5b618e5bc69 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:39.864245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f342472-9a46-4d16-9ee3-a5b618e5bc69 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:39.871381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:39.872199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:39.872562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:39.873081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] PATH_INFO: `/v3/users/2837531809ef457997a780caf0e04540` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:39.874374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2837531809ef457997a780caf0e04540)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:39.876450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:40.045342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2837531809ef457997a780caf0e04540'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:40.046908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:40.062869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2837531809ef457997a780caf0e04540'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:40.063901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2837531809ef457997a780caf0e04540'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:40.085182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2837531809ef457997a780caf0e04540'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:40.167587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2837531809ef457997a780caf0e04540'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:40.176592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1073/2113] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:30:39 2022] DELETE /identity/v3/users/2837531809ef457997a780caf0e04540 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:40.182929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4b8a1-6927-4e6e-8fd0-c2e1ae3d1adb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:40.183589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4b8a1-6927-4e6e-8fd0-c2e1ae3d1adb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:40.184166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4b8a1-6927-4e6e-8fd0-c2e1ae3d1adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:40.207132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8ad4b8a1-6927-4e6e-8fd0-c2e1ae3d1adb None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:40.208383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8ad4b8a1-6927-4e6e-8fd0-c2e1ae3d1adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:40.291066 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1041/2114] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:40.300861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:40.302523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:40.398843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d5d00-1ca6-44c8-a2bd-b7747dc86601 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:40.403679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b5078-1b4b-433f-b7c8-e24fc95017a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:40.404900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b5078-1b4b-433f-b7c8-e24fc95017a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:40.404900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b5078-1b4b-433f-b7c8-e24fc95017a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:40.405222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b5078-1b4b-433f-b7c8-e24fc95017a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:40.426002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b5078-1b4b-433f-b7c8-e24fc95017a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:40.427632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b5078-1b4b-433f-b7c8-e24fc95017a8 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:40.444642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1074/2115] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:41.077375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4b8a1-6927-4e6e-8fd0-c2e1ae3d1adb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:41.079592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8ad4b8a1-6927-4e6e-8fd0-c2e1ae3d1adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:41.117028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4b8a1-6927-4e6e-8fd0-c2e1ae3d1adb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:41.127181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:41.127930 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:41.128166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:41.129642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] PATH_INFO: `/v3/projects/5feed1ec709e49c48bb9be30d3451c3b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:41.133834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5feed1ec709e49c48bb9be30d3451c3b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:41.136276 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:41.294720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5feed1ec709e49c48bb9be30d3451c3b', 'request_id': 'req-f6e877bf-9c49-4993-b8a6-3782d8517806'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:41.318334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5feed1ec709e49c48bb9be30d3451c3b', 'request_id': 'req-f6e877bf-9c49-4993-b8a6-3782d8517806'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:41.330772 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5feed1ec709e49c48bb9be30d3451c3b', 'request_id': 'req-f6e877bf-9c49-4993-b8a6-3782d8517806'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:41.439531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] The token cache is being invalidate because project 5feed1ec709e49c48bb9be30d3451c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:30:41.439826 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:41.440510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:41.444386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1042/2116] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:30:41 2022] DELETE /identity/v3/projects/5feed1ec709e49c48bb9be30d3451c3b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:41.448490 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b5078-1b4b-433f-b7c8-e24fc95017a8 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:41.458597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b5078-1b4b-433f-b7c8-e24fc95017a8 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:41.539747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:41.541332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:41.541899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:41.542530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] PATH_INFO: `/v3/users/1cda45eb2a4b4ef5a8589b6fed5a0977` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:41.544245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cda45eb2a4b4ef5a8589b6fed5a0977)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:41.547707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:41.749533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cda45eb2a4b4ef5a8589b6fed5a0977'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:41.763405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cda45eb2a4b4ef5a8589b6fed5a0977'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:41.764924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cda45eb2a4b4ef5a8589b6fed5a0977'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:41.772735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:41.778783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cda45eb2a4b4ef5a8589b6fed5a0977'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:41.891444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-88267377-e21e-486e-9ede-5ae611867d4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cda45eb2a4b4ef5a8589b6fed5a0977'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:41.897882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1075/2117] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:30:41 2022] DELETE /identity/v3/users/1cda45eb2a4b4ef5a8589b6fed5a0977 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:30:42.508633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:42.514855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:42.534940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e877bf-9c49-4993-b8a6-3782d8517806 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:42.639530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:42.640384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:42.640896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:42.641987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] PATH_INFO: `/v3/projects/43c9e6be6b3e4d28a6f06274a8c6c1dd` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:42.643607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c9e6be6b3e4d28a6f06274a8c6c1dd)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:42.646193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:42.682125 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:42.800261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c9e6be6b3e4d28a6f06274a8c6c1dd', 'request_id': 'req-ce162d39-2dc4-4a10-a4ad-a238690b0f35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:42.824810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c9e6be6b3e4d28a6f06274a8c6c1dd', 'request_id': 'req-ce162d39-2dc4-4a10-a4ad-a238690b0f35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:42.834268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c9e6be6b3e4d28a6f06274a8c6c1dd', 'request_id': 'req-ce162d39-2dc4-4a10-a4ad-a238690b0f35'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:42.936860 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] The token cache is being invalidate because project 43c9e6be6b3e4d28a6f06274a8c6c1dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:30:42.936860 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:42.936860 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ce162d39-2dc4-4a10-a4ad-a238690b0f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:42.949121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02956f26-68b6-4978-93ce-c59240550a9e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:42.949784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02956f26-68b6-4978-93ce-c59240550a9e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:42.950392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02956f26-68b6-4978-93ce-c59240550a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:42.960005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1043/2118] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:30:42 2022] DELETE /identity/v3/projects/43c9e6be6b3e4d28a6f06274a8c6c1dd => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:42.993258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-02956f26-68b6-4978-93ce-c59240550a9e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:42.995905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-02956f26-68b6-4978-93ce-c59240550a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:43.166724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1076/2119] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:43.174178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371ecab-3c03-441a-a83d-ab58e2414813 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.174669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371ecab-3c03-441a-a83d-ab58e2414813 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.174813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371ecab-3c03-441a-a83d-ab58e2414813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:43.208189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3371ecab-3c03-441a-a83d-ab58e2414813 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:43.209437 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3371ecab-3c03-441a-a83d-ab58e2414813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:43.230778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1044/2120] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:43.235273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02956f26-68b6-4978-93ce-c59240550a9e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:43.257997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02956f26-68b6-4978-93ce-c59240550a9e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:43.291166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd0845-6c2d-41fd-ad1c-d312043b0f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:43.293283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd0845-6c2d-41fd-ad1c-d312043b0f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.293283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd0845-6c2d-41fd-ad1c-d312043b0f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.293283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd0845-6c2d-41fd-ad1c-d312043b0f63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:43.294421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd0845-6c2d-41fd-ad1c-d312043b0f63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:30:43.294804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd0845-6c2d-41fd-ad1c-d312043b0f63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:43.296973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd0845-6c2d-41fd-ad1c-d312043b0f63 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:43.317375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1077/2121] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:30:43.319285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371ecab-3c03-441a-a83d-ab58e2414813 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:43.341337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371ecab-3c03-441a-a83d-ab58e2414813 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:43.349866 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77985007-b84b-47b2-b66d-7a4bc5ae62dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:43.350807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77985007-b84b-47b2-b66d-7a4bc5ae62dc admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.351290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77985007-b84b-47b2-b66d-7a4bc5ae62dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.351848 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77985007-b84b-47b2-b66d-7a4bc5ae62dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:43.353127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77985007-b84b-47b2-b66d-7a4bc5ae62dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:43.359728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77985007-b84b-47b2-b66d-7a4bc5ae62dc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:43.484886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1045/2122] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:30:43 2022] POST /identity/v3/projects => generated 409 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:43.488964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd0845-6c2d-41fd-ad1c-d312043b0f63 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:43.503716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd0845-6c2d-41fd-ad1c-d312043b0f63 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:43.601024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eb0be-acce-41c3-89f4-fbd67200451d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:43.601719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eb0be-acce-41c3-89f4-fbd67200451d admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.602008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eb0be-acce-41c3-89f4-fbd67200451d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.602292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eb0be-acce-41c3-89f4-fbd67200451d admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:43.603107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eb0be-acce-41c3-89f4-fbd67200451d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:43.606434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eb0be-acce-41c3-89f4-fbd67200451d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:43.700959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1078/2123] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:30:43 2022] POST /identity/v3/users => generated 305 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:43.704994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77985007-b84b-47b2-b66d-7a4bc5ae62dc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:43.720271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77985007-b84b-47b2-b66d-7a4bc5ae62dc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:43.730874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dcc65-314f-47f1-bcb5-35d59f7ed1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:43.732047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcc65-314f-47f1-bcb5-35d59f7ed1dc admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.732813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcc65-314f-47f1-bcb5-35d59f7ed1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.733481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcc65-314f-47f1-bcb5-35d59f7ed1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:43.734590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dcc65-314f-47f1-bcb5-35d59f7ed1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:43.737651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dcc65-314f-47f1-bcb5-35d59f7ed1dc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:43.754886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1046/2124] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:43.758678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eb0be-acce-41c3-89f4-fbd67200451d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:43.777179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eb0be-acce-41c3-89f4-fbd67200451d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:43.789842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085340e-bbea-40af-8dcf-38689f0a69aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:43.790958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085340e-bbea-40af-8dcf-38689f0a69aa admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.791546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085340e-bbea-40af-8dcf-38689f0a69aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.792091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085340e-bbea-40af-8dcf-38689f0a69aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:43.793053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085340e-bbea-40af-8dcf-38689f0a69aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:43.795059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085340e-bbea-40af-8dcf-38689f0a69aa admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:43.811123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1079/2125] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:30:43.815797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dcc65-314f-47f1-bcb5-35d59f7ed1dc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:43.835670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dcc65-314f-47f1-bcb5-35d59f7ed1dc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:43.846523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ae027-7975-49b0-ad6c-2f180499e270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:43.846866 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ae027-7975-49b0-ad6c-2f180499e270 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.847154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ae027-7975-49b0-ad6c-2f180499e270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.847441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ae027-7975-49b0-ad6c-2f180499e270 admin admin] PATH_INFO: `/v3/projects/b5a12cc7d3e948499c57efaaeb6cd423/users/7434642e30734b34b6e917fec2ea6e9d/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:43.889684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ae027-7975-49b0-ad6c-2f180499e270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a12cc7d3e948499c57efaaeb6cd423, user_id=7434642e30734b34b6e917fec2ea6e9d, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:43.892419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ae027-7975-49b0-ad6c-2f180499e270 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:43.983893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1047/2126] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:43 2022] PUT /identity/v3/projects/b5a12cc7d3e948499c57efaaeb6cd423/users/7434642e30734b34b6e917fec2ea6e9d/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:43.996377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad3c06-6850-4aa2-a2fd-47fe57ba89ac None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.996897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad3c06-6850-4aa2-a2fd-47fe57ba89ac None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.996897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad3c06-6850-4aa2-a2fd-47fe57ba89ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:44.037191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-22ad3c06-6850-4aa2-a2fd-47fe57ba89ac None None] MFA Rules not processed for user `7434642e30734b34b6e917fec2ea6e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:44.038494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-22ad3c06-6850-4aa2-a2fd-47fe57ba89ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:44.110718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:44.147890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1080/2127] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:30:44.148115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ae027-7975-49b0-ad6c-2f180499e270 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:44.268612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ae027-7975-49b0-ad6c-2f180499e270 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:44.278024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c89c27-b51d-4718-80c7-9a40dd395192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:44.279659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c89c27-b51d-4718-80c7-9a40dd395192 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:44.280121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c89c27-b51d-4718-80c7-9a40dd395192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:44.280527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c89c27-b51d-4718-80c7-9a40dd395192 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:44.281376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c89c27-b51d-4718-80c7-9a40dd395192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:44.283605 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c89c27-b51d-4718-80c7-9a40dd395192 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:44.406820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1048/2128] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:30:44 2022] POST /identity/v3/projects => generated 407 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:44.411611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad3c06-6850-4aa2-a2fd-47fe57ba89ac None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:44.446068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad3c06-6850-4aa2-a2fd-47fe57ba89ac None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:44.663773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bc338-95ea-48cc-817b-cf6dfa4cb73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:44.664415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bc338-95ea-48cc-817b-cf6dfa4cb73a admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:44.664697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bc338-95ea-48cc-817b-cf6dfa4cb73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:44.664969 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bc338-95ea-48cc-817b-cf6dfa4cb73a admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:44.666325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bc338-95ea-48cc-817b-cf6dfa4cb73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:44.671045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bc338-95ea-48cc-817b-cf6dfa4cb73a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:44.763416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1081/2129] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:30:44 2022] POST /identity/v3/users => generated 310 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:44.767292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c89c27-b51d-4718-80c7-9a40dd395192 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:44.782290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c89c27-b51d-4718-80c7-9a40dd395192 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:44.788542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9577c-8a9a-4a35-9763-843bec4cd9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:44.789533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9577c-8a9a-4a35-9763-843bec4cd9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:44.790067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9577c-8a9a-4a35-9763-843bec4cd9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:44.790586 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9577c-8a9a-4a35-9763-843bec4cd9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:44.791553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9577c-8a9a-4a35-9763-843bec4cd9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:44.793610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9577c-8a9a-4a35-9763-843bec4cd9f2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:44.812108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1049/2130] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:44.814553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bc338-95ea-48cc-817b-cf6dfa4cb73a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:44.829628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bc338-95ea-48cc-817b-cf6dfa4cb73a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:44.874513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88703e9-0628-4ada-84f3-801d7c9962d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:44.877826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88703e9-0628-4ada-84f3-801d7c9962d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:44.878118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88703e9-0628-4ada-84f3-801d7c9962d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:44.878403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88703e9-0628-4ada-84f3-801d7c9962d3 admin admin] PATH_INFO: `/v3/projects/cc0c4028572f4cc299b87d3409418353/users/b1592869c7804c83a1ccb6da0f2e99e0/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:44.907078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88703e9-0628-4ada-84f3-801d7c9962d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0c4028572f4cc299b87d3409418353, user_id=b1592869c7804c83a1ccb6da0f2e99e0, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:44.910326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88703e9-0628-4ada-84f3-801d7c9962d3 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:45.031979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1082/2131] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:44 2022] PUT /identity/v3/projects/cc0c4028572f4cc299b87d3409418353/users/b1592869c7804c83a1ccb6da0f2e99e0/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:45.036413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9577c-8a9a-4a35-9763-843bec4cd9f2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:45.054008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9577c-8a9a-4a35-9763-843bec4cd9f2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:45.102085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:45.260788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21da2e4-79be-4a56-99a1-a036d90c9e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:45.261879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da2e4-79be-4a56-99a1-a036d90c9e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:45.262353 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da2e4-79be-4a56-99a1-a036d90c9e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:45.262869 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da2e4-79be-4a56-99a1-a036d90c9e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:45.265618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21da2e4-79be-4a56-99a1-a036d90c9e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:45.268457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21da2e4-79be-4a56-99a1-a036d90c9e17 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:45.305503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1050/2132] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:30:45.309269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88703e9-0628-4ada-84f3-801d7c9962d3 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:45.342689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88703e9-0628-4ada-84f3-801d7c9962d3 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:45.357796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cd7ce-3777-421d-b081-0a99b28c6834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:45.358812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd7ce-3777-421d-b081-0a99b28c6834 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:45.359257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd7ce-3777-421d-b081-0a99b28c6834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:45.369045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd7ce-3777-421d-b081-0a99b28c6834 admin admin] PATH_INFO: `/v3/projects/cc0c4028572f4cc299b87d3409418353/users/b1592869c7804c83a1ccb6da0f2e99e0/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:45.371656 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cd7ce-3777-421d-b081-0a99b28c6834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0c4028572f4cc299b87d3409418353, user_id=b1592869c7804c83a1ccb6da0f2e99e0, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:45.375684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cd7ce-3777-421d-b081-0a99b28c6834 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:45.395733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1083/2133] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:45 2022] PUT /identity/v3/projects/cc0c4028572f4cc299b87d3409418353/users/b1592869c7804c83a1ccb6da0f2e99e0/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:45.407379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41acd8-9e3e-4e65-996b-4bc8d390c973 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:45.408171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41acd8-9e3e-4e65-996b-4bc8d390c973 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:45.408575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41acd8-9e3e-4e65-996b-4bc8d390c973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:45.480039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2c41acd8-9e3e-4e65-996b-4bc8d390c973 None None] MFA Rules not processed for user `b1592869c7804c83a1ccb6da0f2e99e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:45.481379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2c41acd8-9e3e-4e65-996b-4bc8d390c973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:45.814682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1051/2134] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:45 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:45.980033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cd7ce-3777-421d-b081-0a99b28c6834 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:45.981849 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c52cd7ce-3777-421d-b081-0a99b28c6834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:46.086017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cd7ce-3777-421d-b081-0a99b28c6834 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:46.097431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51ecc6-816e-4cc6-a844-74e2873f8883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:46.114191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ecc6-816e-4cc6-a844-74e2873f8883 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:46.114500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ecc6-816e-4cc6-a844-74e2873f8883 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:46.114843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ecc6-816e-4cc6-a844-74e2873f8883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:46.145332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51ecc6-816e-4cc6-a844-74e2873f8883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:46.149081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51ecc6-816e-4cc6-a844-74e2873f8883 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:46.167576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1084/2135] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:45 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:46.972949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41acd8-9e3e-4e65-996b-4bc8d390c973 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:46.996371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41acd8-9e3e-4e65-996b-4bc8d390c973 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:47.002678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8376faf-7df9-4ba0-ac26-453b8c144dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:47.003909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8376faf-7df9-4ba0-ac26-453b8c144dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:47.004585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8376faf-7df9-4ba0-ac26-453b8c144dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:47.005075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8376faf-7df9-4ba0-ac26-453b8c144dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:47.229514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8376faf-7df9-4ba0-ac26-453b8c144dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:47.235045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8376faf-7df9-4ba0-ac26-453b8c144dd5 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:47.288820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1052/2136] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:30:49.031324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684eaca1-f708-4179-a95c-bde39982ddc0 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:49.036468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684eaca1-f708-4179-a95c-bde39982ddc0 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:49.036468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684eaca1-f708-4179-a95c-bde39982ddc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:49.065695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-684eaca1-f708-4179-a95c-bde39982ddc0 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:49.067988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-684eaca1-f708-4179-a95c-bde39982ddc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:49.128664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:49.190570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1085/2137] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 28 06:30:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:49.223074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8376faf-7df9-4ba0-ac26-453b8c144dd5 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:49.226261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f8376faf-7df9-4ba0-ac26-453b8c144dd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:49.328468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:49.379657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8376faf-7df9-4ba0-ac26-453b8c144dd5 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:49.388138 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27904bf-cc8f-4114-b999-491b49fae20a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:49.389229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27904bf-cc8f-4114-b999-491b49fae20a service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:49.389229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27904bf-cc8f-4114-b999-491b49fae20a service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:49.393968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27904bf-cc8f-4114-b999-491b49fae20a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:49.410109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27904bf-cc8f-4114-b999-491b49fae20a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:49.419441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27904bf-cc8f-4114-b999-491b49fae20a service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:49.476247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1053/2138] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:30:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:30:49.736560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b46d1-01f4-4cc7-b8ec-4579d0cb28f9 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:49.742579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b46d1-01f4-4cc7-b8ec-4579d0cb28f9 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:49.742579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b46d1-01f4-4cc7-b8ec-4579d0cb28f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:49.772108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-778b46d1-01f4-4cc7-b8ec-4579d0cb28f9 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:49.772108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-778b46d1-01f4-4cc7-b8ec-4579d0cb28f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:49.785567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1086/2139] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:30:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:49.810948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27904bf-cc8f-4114-b999-491b49fae20a service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:49.852491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27904bf-cc8f-4114-b999-491b49fae20a service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:49.866436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b231ae-c18e-426d-a2f8-49b8c4bb10be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:49.867741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b231ae-c18e-426d-a2f8-49b8c4bb10be service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:49.869065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b231ae-c18e-426d-a2f8-49b8c4bb10be service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:49.869732 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b231ae-c18e-426d-a2f8-49b8c4bb10be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:49.898765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b231ae-c18e-426d-a2f8-49b8c4bb10be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:49.903043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b231ae-c18e-426d-a2f8-49b8c4bb10be service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:49.924249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1054/2140] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:30:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:30:50.052152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871ad92-8dc6-489e-99ad-4044d33fd6c1 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:50.053037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871ad92-8dc6-489e-99ad-4044d33fd6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:50.053348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871ad92-8dc6-489e-99ad-4044d33fd6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:50.079871 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b871ad92-8dc6-489e-99ad-4044d33fd6c1 None None] MFA Rules not processed for user `61ad8186d91848bc8be42e3be51e9a75`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:50.081538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b871ad92-8dc6-489e-99ad-4044d33fd6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:50.170877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1087/2141] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:51.387116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e958cca-7ec6-4b41-bd21-8ec1a9d02c45 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:51.389533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e958cca-7ec6-4b41-bd21-8ec1a9d02c45 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:51.389838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e958cca-7ec6-4b41-bd21-8ec1a9d02c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:51.423820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2e958cca-7ec6-4b41-bd21-8ec1a9d02c45 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:51.426589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2e958cca-7ec6-4b41-bd21-8ec1a9d02c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:51.440945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1055/2142] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 28 06:30:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:51.456638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871ad92-8dc6-489e-99ad-4044d33fd6c1 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:51.469919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871ad92-8dc6-489e-99ad-4044d33fd6c1 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:51.476345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9931eb7-746b-44ff-ae52-cd1e33bfe684 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:51.477242 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9931eb7-746b-44ff-ae52-cd1e33bfe684 service swift] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:51.482172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9931eb7-746b-44ff-ae52-cd1e33bfe684 service swift] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:51.482172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9931eb7-746b-44ff-ae52-cd1e33bfe684 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:51.492716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9931eb7-746b-44ff-ae52-cd1e33bfe684 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:51.495346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9931eb7-746b-44ff-ae52-cd1e33bfe684 service swift] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:51.505172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1088/2143] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:30:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:30:52.363077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dd256-39d1-4812-b29f-b3886882b0a8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:52.363374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dd256-39d1-4812-b29f-b3886882b0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:52.363661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dd256-39d1-4812-b29f-b3886882b0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:52.387224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-017dd256-39d1-4812-b29f-b3886882b0a8 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:52.388467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-017dd256-39d1-4812-b29f-b3886882b0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:52.396437 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1056/2144] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:30:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:52.410528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9931eb7-746b-44ff-ae52-cd1e33bfe684 service swift] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:52.422024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9931eb7-746b-44ff-ae52-cd1e33bfe684 service swift] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:52.428156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc1dc0-18a6-466b-a3be-648cc8ed54ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:52.428905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc1dc0-18a6-466b-a3be-648cc8ed54ec service swift] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:52.429043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc1dc0-18a6-466b-a3be-648cc8ed54ec service swift] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:52.429415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc1dc0-18a6-466b-a3be-648cc8ed54ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:52.451054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc1dc0-18a6-466b-a3be-648cc8ed54ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:52.451968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc1dc0-18a6-466b-a3be-648cc8ed54ec service swift] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:52.461910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1089/2145] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:30:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:30:53.204247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ec24-6866-45d9-9ece-8449b2e22877 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:53.204590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ec24-6866-45d9-9ece-8449b2e22877 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:53.204897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ec24-6866-45d9-9ece-8449b2e22877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:53.226137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a9a7ec24-6866-45d9-9ece-8449b2e22877 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:53.227387 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a9a7ec24-6866-45d9-9ece-8449b2e22877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:53.237067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1057/2146] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 28 06:30:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:53.251947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc1dc0-18a6-466b-a3be-648cc8ed54ec service swift] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:53.270272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc1dc0-18a6-466b-a3be-648cc8ed54ec service swift] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:53.276910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b761b0-94a0-45c2-a6cb-2c90bce5d577 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:53.283120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b761b0-94a0-45c2-a6cb-2c90bce5d577 service swift] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:53.283120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b761b0-94a0-45c2-a6cb-2c90bce5d577 service swift] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:53.283120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b761b0-94a0-45c2-a6cb-2c90bce5d577 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:53.288921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b761b0-94a0-45c2-a6cb-2c90bce5d577 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:53.291129 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b761b0-94a0-45c2-a6cb-2c90bce5d577 service swift] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:53.301859 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1090/2147] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:30:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:30:57.268891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7ec24-6866-45d9-9ece-8449b2e22877 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:57.274103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a9a7ec24-6866-45d9-9ece-8449b2e22877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:57.379791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7ec24-6866-45d9-9ece-8449b2e22877 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:57.386467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:57.387305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:57.387673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:57.388020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] PATH_INFO: `/v3/users/85d2763ce8a64f128f28ae36a50a8890` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:57.389822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d2763ce8a64f128f28ae36a50a8890)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:57.392251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:57.678443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d2763ce8a64f128f28ae36a50a8890'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:57.694711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d2763ce8a64f128f28ae36a50a8890'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:57.702239 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d2763ce8a64f128f28ae36a50a8890'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:57.760636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d2763ce8a64f128f28ae36a50a8890'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:57.762407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d2763ce8a64f128f28ae36a50a8890'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:57.769146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1058/2148] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:30:57 2022] DELETE /identity/v3/users/85d2763ce8a64f128f28ae36a50a8890 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:57.773762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4b12d-e9ef-4d11-b0fb-3e43209d0414 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:57.774236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4b12d-e9ef-4d11-b0fb-3e43209d0414 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:57.774751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4b12d-e9ef-4d11-b0fb-3e43209d0414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:57.795124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-0dd4b12d-e9ef-4d11-b0fb-3e43209d0414 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:57.796397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0dd4b12d-e9ef-4d11-b0fb-3e43209d0414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:57.872982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1091/2149] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:57.880321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:57.881901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:57.899122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:57.963255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b797c68-d668-4aa5-baa1-6e3c0371c81e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:57.967693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e767c-a00a-453b-be33-4133379d6dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:57.968404 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e767c-a00a-453b-be33-4133379d6dab service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:57.968757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e767c-a00a-453b-be33-4133379d6dab service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:57.969169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e767c-a00a-453b-be33-4133379d6dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:57.987478 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e767c-a00a-453b-be33-4133379d6dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:57.989255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e767c-a00a-453b-be33-4133379d6dab service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:58.006600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1059/2150] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:58.340926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4b12d-e9ef-4d11-b0fb-3e43209d0414 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:58.343427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0dd4b12d-e9ef-4d11-b0fb-3e43209d0414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:58.357862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4b12d-e9ef-4d11-b0fb-3e43209d0414 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:58.363092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:58.369133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:58.369133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:58.369133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] PATH_INFO: `/v3/projects/08e27723b1224589a896eab04740dffc` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:58.369133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e27723b1224589a896eab04740dffc)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:58.369133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:58.537178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:58.548684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e27723b1224589a896eab04740dffc', 'request_id': 'req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:58.557737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e27723b1224589a896eab04740dffc', 'request_id': 'req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:58.623489 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e27723b1224589a896eab04740dffc', 'request_id': 'req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:58.637649 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] The token cache is being invalidate because project 08e27723b1224589a896eab04740dffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:30:58.637984 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:58.641057 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c99058ef-a8ce-4cd1-b22c-83f4cd594eee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:58.645714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1092/2151] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:30:58 2022] DELETE /identity/v3/projects/08e27723b1224589a896eab04740dffc => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:58.647902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2287549-7ef8-4163-a2ee-097b86f6439e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:58.648382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2287549-7ef8-4163-a2ee-097b86f6439e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:58.648484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2287549-7ef8-4163-a2ee-097b86f6439e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:58.684206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e2287549-7ef8-4163-a2ee-097b86f6439e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:58.685193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e2287549-7ef8-4163-a2ee-097b86f6439e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:58.751470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d653eeb-08b9-4d90-895b-61eb1db8feca None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:58.752228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d653eeb-08b9-4d90-895b-61eb1db8feca None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:58.754016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d653eeb-08b9-4d90-895b-61eb1db8feca None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:58.757550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1093/2152] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:30:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:30:58.758640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1060/2153] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:58.767253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953a7d3-ebe6-4954-9182-68ec83ccb5d9 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:58.767546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953a7d3-ebe6-4954-9182-68ec83ccb5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:58.767830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953a7d3-ebe6-4954-9182-68ec83ccb5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:58.790189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-8953a7d3-ebe6-4954-9182-68ec83ccb5d9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:58.792127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8953a7d3-ebe6-4954-9182-68ec83ccb5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:58.797627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4f91a-cff9-422f-ae94-34865c38e7a6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:58.797922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4f91a-cff9-422f-ae94-34865c38e7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:58.798322 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4f91a-cff9-422f-ae94-34865c38e7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:58.803715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1094/2154] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:58.809300 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953a7d3-ebe6-4954-9182-68ec83ccb5d9 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:58.822236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953a7d3-ebe6-4954-9182-68ec83ccb5d9 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:58.828951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2fd4f91a-cff9-422f-ae94-34865c38e7a6 None None] MFA Rules not processed for user `244a8787df7641f2bac9b51213ddb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:58.830338 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2fd4f91a-cff9-422f-ae94-34865c38e7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:58.831293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606fcf1-5998-40f5-ae0d-b8da77e51e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:58.832986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606fcf1-5998-40f5-ae0d-b8da77e51e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:58.833506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606fcf1-5998-40f5-ae0d-b8da77e51e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:58.839432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606fcf1-5998-40f5-ae0d-b8da77e51e97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:58.839432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606fcf1-5998-40f5-ae0d-b8da77e51e97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:30:58.839432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606fcf1-5998-40f5-ae0d-b8da77e51e97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:58.840270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606fcf1-5998-40f5-ae0d-b8da77e51e97 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:58.874711 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1095/2155] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:30:58.875912 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606fcf1-5998-40f5-ae0d-b8da77e51e97 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:58.892815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606fcf1-5998-40f5-ae0d-b8da77e51e97 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:58.898200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdeeb1-0ad9-469c-832d-fbd5d0858512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:58.899028 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdeeb1-0ad9-469c-832d-fbd5d0858512 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:58.900356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdeeb1-0ad9-469c-832d-fbd5d0858512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:58.900807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdeeb1-0ad9-469c-832d-fbd5d0858512 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:58.901736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdeeb1-0ad9-469c-832d-fbd5d0858512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:58.905504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdeeb1-0ad9-469c-832d-fbd5d0858512 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:58.949337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1061/2156] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:30:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:30:58.950269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4f91a-cff9-422f-ae94-34865c38e7a6 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:58.953201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2fd4f91a-cff9-422f-ae94-34865c38e7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:58.975970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4f91a-cff9-422f-ae94-34865c38e7a6 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:58.982474 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:58.983211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:58.983562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:58.984097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] PATH_INFO: `/v3/users/eb2a7c1097e0421492e8460e9b64a27b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:58.985408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2a7c1097e0421492e8460e9b64a27b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:58.987513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:59.039737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1096/2157] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:30:58 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:59.041686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdeeb1-0ad9-469c-832d-fbd5d0858512 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:59.045120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-acbdeeb1-0ad9-469c-832d-fbd5d0858512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:59.137345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdeeb1-0ad9-469c-832d-fbd5d0858512 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:59.141639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782a39b-79bb-488c-882e-864e59e44288 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:59.142309 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a39b-79bb-488c-882e-864e59e44288 service placement] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.142576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a39b-79bb-488c-882e-864e59e44288 service placement] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.142891 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a39b-79bb-488c-882e-864e59e44288 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.152901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782a39b-79bb-488c-882e-864e59e44288 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:59.154563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782a39b-79bb-488c-882e-864e59e44288 service placement] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:59.165967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1097/2158] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:59.166815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782a39b-79bb-488c-882e-864e59e44288 service placement] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:59.194782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:59.197899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2a7c1097e0421492e8460e9b64a27b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:59.224713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2a7c1097e0421492e8460e9b64a27b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:59.233987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2a7c1097e0421492e8460e9b64a27b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:59.281151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782a39b-79bb-488c-882e-864e59e44288 service placement] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:59.286631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2a7c1097e0421492e8460e9b64a27b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:59.287492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0af7f-d149-40ca-afe7-9791a0922cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:59.287945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2a7c1097e0421492e8460e9b64a27b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:30:59.288496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0af7f-d149-40ca-afe7-9791a0922cef admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.288957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0af7f-d149-40ca-afe7-9791a0922cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.289461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0af7f-d149-40ca-afe7-9791a0922cef admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.290025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0af7f-d149-40ca-afe7-9791a0922cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:59.293147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0af7f-d149-40ca-afe7-9791a0922cef admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:59.300121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1062/2159] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:30:58 2022] DELETE /identity/v3/users/eb2a7c1097e0421492e8460e9b64a27b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:59.366444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1098/2160] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:30:59 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:30:59.371390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:59.372623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:59.470195 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50576b-9f51-4aa2-874e-6e8d2d496929 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:59.494808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780b23-abd7-413d-939c-91b3f285eea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:59.494808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780b23-abd7-413d-939c-91b3f285eea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.494808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780b23-abd7-413d-939c-91b3f285eea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.494808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780b23-abd7-413d-939c-91b3f285eea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.495181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff780b23-abd7-413d-939c-91b3f285eea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:59.495181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff780b23-abd7-413d-939c-91b3f285eea5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:59.505347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1063/2161] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:30:59.509961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0af7f-d149-40ca-afe7-9791a0922cef admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:59.515351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:59.520828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0af7f-d149-40ca-afe7-9791a0922cef admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:59.526518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbe103-ead5-4b0e-809e-85e6454f0bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:59.528775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbe103-ead5-4b0e-809e-85e6454f0bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.528775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbe103-ead5-4b0e-809e-85e6454f0bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.528775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbe103-ead5-4b0e-809e-85e6454f0bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.528775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbe103-ead5-4b0e-809e-85e6454f0bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:59.530513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbe103-ead5-4b0e-809e-85e6454f0bc6 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:59.541688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1099/2162] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:30:59.546260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780b23-abd7-413d-939c-91b3f285eea5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:59.556326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780b23-abd7-413d-939c-91b3f285eea5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:59.562773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1347687-c56d-4139-b390-2ca719f86d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:59.562773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1347687-c56d-4139-b390-2ca719f86d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.562773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1347687-c56d-4139-b390-2ca719f86d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.562773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1347687-c56d-4139-b390-2ca719f86d48 admin admin] PATH_INFO: `/v3/projects/f33d1845b3b643b99b066eaeee429e12/users/97547d56a8684009a794a42ad1c5f489/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.577467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1347687-c56d-4139-b390-2ca719f86d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33d1845b3b643b99b066eaeee429e12, user_id=97547d56a8684009a794a42ad1c5f489, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:59.580228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1347687-c56d-4139-b390-2ca719f86d48 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:59.655420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbe103-ead5-4b0e-809e-85e6454f0bc6 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:59.658322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d0cbe103-ead5-4b0e-809e-85e6454f0bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:59.679597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1064/2163] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:30:59 2022] PUT /identity/v3/projects/f33d1845b3b643b99b066eaeee429e12/users/97547d56a8684009a794a42ad1c5f489/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:30:59.683773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036b959-0d5f-4eb5-b154-33c2c685a12f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.684352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036b959-0d5f-4eb5-b154-33c2c685a12f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.684811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036b959-0d5f-4eb5-b154-33c2c685a12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.695933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbe103-ead5-4b0e-809e-85e6454f0bc6 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:59.715661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-9036b959-0d5f-4eb5-b154-33c2c685a12f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:59.719911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9036b959-0d5f-4eb5-b154-33c2c685a12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:59.791963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:59.795386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.795787 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.796150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] PATH_INFO: `/v3/projects/22a64b6b084848ec89599880170c719e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.798061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a64b6b084848ec89599880170c719e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:59.800422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:59.860495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1065/2164] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:30:59.871511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036b959-0d5f-4eb5-b154-33c2c685a12f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:59.873427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9036b959-0d5f-4eb5-b154-33c2c685a12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:59.916982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:59.941100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036b959-0d5f-4eb5-b154-33c2c685a12f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:59.945242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e91f4d-0289-469f-bc99-7f7ddac9fee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:59.945951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91f4d-0289-469f-bc99-7f7ddac9fee6 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.946301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91f4d-0289-469f-bc99-7f7ddac9fee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.947234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91f4d-0289-469f-bc99-7f7ddac9fee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.957978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e91f4d-0289-469f-bc99-7f7ddac9fee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:59.959585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e91f4d-0289-469f-bc99-7f7ddac9fee6 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:59.971609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1066/2165] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:31:00.006872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a64b6b084848ec89599880170c719e', 'request_id': 'req-d144fa5b-81ef-4062-b339-de2d508974de'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:00.014237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a64b6b084848ec89599880170c719e', 'request_id': 'req-d144fa5b-81ef-4062-b339-de2d508974de'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:00.114601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a64b6b084848ec89599880170c719e', 'request_id': 'req-d144fa5b-81ef-4062-b339-de2d508974de'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:00.131078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] The token cache is being invalidate because project 22a64b6b084848ec89599880170c719e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:31:00.131577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:00.132516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d144fa5b-81ef-4062-b339-de2d508974de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:00.135428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1100/2166] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:30:59 2022] DELETE /identity/v3/projects/22a64b6b084848ec89599880170c719e => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:31:00.140462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849a65-a344-405c-b886-a741d87a4972 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:00.140627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849a65-a344-405c-b886-a741d87a4972 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:00.140839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849a65-a344-405c-b886-a741d87a4972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:00.172632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-74849a65-a344-405c-b886-a741d87a4972 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:00.175220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-74849a65-a344-405c-b886-a741d87a4972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:00.267409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1067/2167] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:00.276473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50e024-f905-456b-8001-1bf18304936d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:00.277347 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50e024-f905-456b-8001-1bf18304936d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:00.277347 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50e024-f905-456b-8001-1bf18304936d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:00.305209 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-cc50e024-f905-456b-8001-1bf18304936d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:00.307770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-cc50e024-f905-456b-8001-1bf18304936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:00.319030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1101/2168] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:00.323891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74849a65-a344-405c-b886-a741d87a4972 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:00.334154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74849a65-a344-405c-b886-a741d87a4972 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:00.340221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784217d-55a7-4620-b8a7-0c029ee220b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:00.341163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784217d-55a7-4620-b8a7-0c029ee220b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:00.341622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784217d-55a7-4620-b8a7-0c029ee220b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:00.342054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784217d-55a7-4620-b8a7-0c029ee220b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:00.342908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784217d-55a7-4620-b8a7-0c029ee220b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:31:00.343390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784217d-55a7-4620-b8a7-0c029ee220b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:00.345207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784217d-55a7-4620-b8a7-0c029ee220b3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:00.401072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1068/2169] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:31:00.405501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50e024-f905-456b-8001-1bf18304936d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:00.418276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50e024-f905-456b-8001-1bf18304936d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:00.425468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ec485-1c3f-4aa9-ad54-eabb0934752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:00.426338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ec485-1c3f-4aa9-ad54-eabb0934752c admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:00.426725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ec485-1c3f-4aa9-ad54-eabb0934752c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:00.427163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ec485-1c3f-4aa9-ad54-eabb0934752c admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:00.428193 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ec485-1c3f-4aa9-ad54-eabb0934752c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:00.430008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ec485-1c3f-4aa9-ad54-eabb0934752c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:00.533841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1102/2170] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:31:00 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:00.538631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784217d-55a7-4620-b8a7-0c029ee220b3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:00.549821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784217d-55a7-4620-b8a7-0c029ee220b3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:00.624919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21c055-3ace-43f5-b36e-25781181a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:00.626116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21c055-3ace-43f5-b36e-25781181a85a admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:00.626662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21c055-3ace-43f5-b36e-25781181a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:00.627201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21c055-3ace-43f5-b36e-25781181a85a admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:00.628684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21c055-3ace-43f5-b36e-25781181a85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:00.632312 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21c055-3ace-43f5-b36e-25781181a85a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:00.724965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1069/2171] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:31:00 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:00.730007 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ec485-1c3f-4aa9-ad54-eabb0934752c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:00.743394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ec485-1c3f-4aa9-ad54-eabb0934752c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:00.749397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbad42d-b435-4262-9c5a-c0760288a4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:00.750153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbad42d-b435-4262-9c5a-c0760288a4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:00.750920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbad42d-b435-4262-9c5a-c0760288a4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:00.750920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbad42d-b435-4262-9c5a-c0760288a4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:00.751605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbad42d-b435-4262-9c5a-c0760288a4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:00.753441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbad42d-b435-4262-9c5a-c0760288a4e4 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:00.764958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1103/2172] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:00.769156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21c055-3ace-43f5-b36e-25781181a85a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:00.781988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21c055-3ace-43f5-b36e-25781181a85a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:00.786831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f8c61-6960-4ffc-b5bb-98890818d089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:00.787526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f8c61-6960-4ffc-b5bb-98890818d089 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:00.787819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f8c61-6960-4ffc-b5bb-98890818d089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:00.788107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f8c61-6960-4ffc-b5bb-98890818d089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:00.788817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f8c61-6960-4ffc-b5bb-98890818d089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:00.791026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f8c61-6960-4ffc-b5bb-98890818d089 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:00.803078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1070/2173] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:00.807951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbad42d-b435-4262-9c5a-c0760288a4e4 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:00.820076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbad42d-b435-4262-9c5a-c0760288a4e4 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:00.826427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0f3e0-c48a-4d8c-87ff-ed3ac35fb10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:00.827109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0f3e0-c48a-4d8c-87ff-ed3ac35fb10e admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:00.827594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0f3e0-c48a-4d8c-87ff-ed3ac35fb10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:00.828058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0f3e0-c48a-4d8c-87ff-ed3ac35fb10e admin admin] PATH_INFO: `/v3/projects/b3130ffab0b24eed8e2e3c4bff28770c/users/2a296d6aea5041ff943f760d79084b1b/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:00.847785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0f3e0-c48a-4d8c-87ff-ed3ac35fb10e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3130ffab0b24eed8e2e3c4bff28770c, user_id=2a296d6aea5041ff943f760d79084b1b, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:00.850940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0f3e0-c48a-4d8c-87ff-ed3ac35fb10e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:00.957800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1104/2174] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:00 2022] PUT /identity/v3/projects/b3130ffab0b24eed8e2e3c4bff28770c/users/2a296d6aea5041ff943f760d79084b1b/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:00.968773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49b6d-47b9-4cbc-9803-29a33cf230ae None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:00.969097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49b6d-47b9-4cbc-9803-29a33cf230ae None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:00.969516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49b6d-47b9-4cbc-9803-29a33cf230ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:01.008181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-3bf49b6d-47b9-4cbc-9803-29a33cf230ae None None] MFA Rules not processed for user `2a296d6aea5041ff943f760d79084b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:01.010518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3bf49b6d-47b9-4cbc-9803-29a33cf230ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:01.069110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:01.117588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1071/2175] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:31:01.132593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0f3e0-c48a-4d8c-87ff-ed3ac35fb10e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:01.134656 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d0e0f3e0-c48a-4d8c-87ff-ed3ac35fb10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:01.178264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:01.284379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0f3e0-c48a-4d8c-87ff-ed3ac35fb10e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:01.289780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760924-9f96-4d17-810d-7011dabb405f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:01.291231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760924-9f96-4d17-810d-7011dabb405f service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:01.291764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760924-9f96-4d17-810d-7011dabb405f service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:01.292103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760924-9f96-4d17-810d-7011dabb405f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:01.306729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b760924-9f96-4d17-810d-7011dabb405f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:01.312979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b760924-9f96-4d17-810d-7011dabb405f service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:01.336918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1105/2176] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:01.367750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf49b6d-47b9-4cbc-9803-29a33cf230ae None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:01.490725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf49b6d-47b9-4cbc-9803-29a33cf230ae None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:01.508499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:01.510130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:01.510658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:01.512652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] PATH_INFO: `/v3/users/2a296d6aea5041ff943f760d79084b1b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:01.514134 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a296d6aea5041ff943f760d79084b1b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:01.516301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:01.691997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760924-9f96-4d17-810d-7011dabb405f service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:01.693627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4b760924-9f96-4d17-810d-7011dabb405f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:01.711238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760924-9f96-4d17-810d-7011dabb405f service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:01.717127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:01.717945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:01.718303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:01.718589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] PATH_INFO: `/v3/users/7434642e30734b34b6e917fec2ea6e9d` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:01.720101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7434642e30734b34b6e917fec2ea6e9d)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:01.722196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:01.757076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a296d6aea5041ff943f760d79084b1b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:01.773762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a296d6aea5041ff943f760d79084b1b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:01.780881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a296d6aea5041ff943f760d79084b1b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:01.856872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a296d6aea5041ff943f760d79084b1b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:01.857770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-0192398c-6655-4a8c-8f72-4e523aba7f00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a296d6aea5041ff943f760d79084b1b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:01.865785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1072/2177] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:31:01 2022] DELETE /identity/v3/users/2a296d6aea5041ff943f760d79084b1b => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:01.867395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c2b2-b2e5-4bbe-8343-6192ca6ca642 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:01.868041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c2b2-b2e5-4bbe-8343-6192ca6ca642 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:01.868631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c2b2-b2e5-4bbe-8343-6192ca6ca642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:01.918046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5dc4c2b2-b2e5-4bbe-8343-6192ca6ca642 None None] MFA Rules not processed for user `97547d56a8684009a794a42ad1c5f489`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:01.918413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7434642e30734b34b6e917fec2ea6e9d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:01.920685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5dc4c2b2-b2e5-4bbe-8343-6192ca6ca642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:01.952159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7434642e30734b34b6e917fec2ea6e9d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:01.953432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7434642e30734b34b6e917fec2ea6e9d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:01.965474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7434642e30734b34b6e917fec2ea6e9d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:02.021849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:02.045285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0812cff9-7fb1-46b9-983f-16e09eb02890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7434642e30734b34b6e917fec2ea6e9d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:02.059833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1106/2178] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:31:01 2022] DELETE /identity/v3/users/7434642e30734b34b6e917fec2ea6e9d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:02.061627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbdca7-e1f3-4297-bda7-fdfe3007ddfe None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:02.062576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbdca7-e1f3-4297-bda7-fdfe3007ddfe None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:02.063586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbdca7-e1f3-4297-bda7-fdfe3007ddfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:02.077689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1073/2179] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:02.079010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cae2ba-8159-4709-8381-04c1aeb753f0 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:02.079944 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cae2ba-8159-4709-8381-04c1aeb753f0 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:02.080230 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cae2ba-8159-4709-8381-04c1aeb753f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:02.129819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d6cae2ba-8159-4709-8381-04c1aeb753f0 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:02.134805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d6cae2ba-8159-4709-8381-04c1aeb753f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:02.136581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-27bbdca7-e1f3-4297-bda7-fdfe3007ddfe None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:02.139189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-27bbdca7-e1f3-4297-bda7-fdfe3007ddfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:02.226730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1074/2180] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:02.228711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cae2ba-8159-4709-8381-04c1aeb753f0 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:02.230173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d6cae2ba-8159-4709-8381-04c1aeb753f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:02.256029 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1107/2181] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:02.257058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbdca7-e1f3-4297-bda7-fdfe3007ddfe None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:02.266784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-27bbdca7-e1f3-4297-bda7-fdfe3007ddfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:02.319046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cae2ba-8159-4709-8381-04c1aeb753f0 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:02.340679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cdc07-1b4d-48fc-a8b2-2235834456ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:02.342250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cdc07-1b4d-48fc-a8b2-2235834456ab service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:02.343047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cdc07-1b4d-48fc-a8b2-2235834456ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:02.343566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cdc07-1b4d-48fc-a8b2-2235834456ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:02.379762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbdca7-e1f3-4297-bda7-fdfe3007ddfe None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:02.386733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b22f0-9edd-4bf9-82a2-68b66c6f2354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:02.387803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b22f0-9edd-4bf9-82a2-68b66c6f2354 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:02.388314 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b22f0-9edd-4bf9-82a2-68b66c6f2354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:02.388690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b22f0-9edd-4bf9-82a2-68b66c6f2354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:02.402987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b22f0-9edd-4bf9-82a2-68b66c6f2354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:02.405551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b22f0-9edd-4bf9-82a2-68b66c6f2354 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:02.422139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1108/2182] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:02.423562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b22f0-9edd-4bf9-82a2-68b66c6f2354 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:02.437011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b22f0-9edd-4bf9-82a2-68b66c6f2354 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:02.442474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6d351-8b97-4b0f-b3c9-4685a8cfc927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:02.443738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6d351-8b97-4b0f-b3c9-4685a8cfc927 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:02.444736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6d351-8b97-4b0f-b3c9-4685a8cfc927 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:02.445372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6d351-8b97-4b0f-b3c9-4685a8cfc927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:02.461290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d6d351-8b97-4b0f-b3c9-4685a8cfc927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:02.465043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d6d351-8b97-4b0f-b3c9-4685a8cfc927 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:02.484420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1109/2183] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:02.521185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cdc07-1b4d-48fc-a8b2-2235834456ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:02.523441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cdc07-1b4d-48fc-a8b2-2235834456ab service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:02.532252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:02.537557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1075/2184] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:03.059672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7c593-b79a-4ec7-b357-79c2879c6d96 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:03.061299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7c593-b79a-4ec7-b357-79c2879c6d96 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:03.061775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7c593-b79a-4ec7-b357-79c2879c6d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:03.084746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-74e7c593-b79a-4ec7-b357-79c2879c6d96 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:03.086389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-74e7c593-b79a-4ec7-b357-79c2879c6d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:03.153764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1110/2185] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:31:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:03.169472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cdc07-1b4d-48fc-a8b2-2235834456ab service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:03.171718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-689cdc07-1b4d-48fc-a8b2-2235834456ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:03.253919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cdc07-1b4d-48fc-a8b2-2235834456ab service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:03.260045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358956c5-4a88-4c6f-9301-1845dee1a84f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:03.260698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358956c5-4a88-4c6f-9301-1845dee1a84f service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:03.260971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358956c5-4a88-4c6f-9301-1845dee1a84f service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:03.261279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358956c5-4a88-4c6f-9301-1845dee1a84f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:03.269972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358956c5-4a88-4c6f-9301-1845dee1a84f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:03.271581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358956c5-4a88-4c6f-9301-1845dee1a84f service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:03.280691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1076/2186] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:31:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:31:03.334432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7c593-b79a-4ec7-b357-79c2879c6d96 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:03.336562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-74e7c593-b79a-4ec7-b357-79c2879c6d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:03.340473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358956c5-4a88-4c6f-9301-1845dee1a84f service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:03.341907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-358956c5-4a88-4c6f-9301-1845dee1a84f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:03.365538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358956c5-4a88-4c6f-9301-1845dee1a84f service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:03.366413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7c593-b79a-4ec7-b357-79c2879c6d96 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:03.374006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:03.374823 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:03.375549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:03.375549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] PATH_INFO: `/v3/projects/b5a12cc7d3e948499c57efaaeb6cd423` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:03.377180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5a12cc7d3e948499c57efaaeb6cd423)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:03.379442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:03.393137 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:03.394196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:03.394678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:03.395151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] PATH_INFO: `/v3/projects/b3130ffab0b24eed8e2e3c4bff28770c` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:03.396978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3130ffab0b24eed8e2e3c4bff28770c)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:03.402143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:03.542830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5a12cc7d3e948499c57efaaeb6cd423', 'request_id': 'req-31b1088f-3436-4857-ab65-16ffa0b1b564'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:03.555468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5a12cc7d3e948499c57efaaeb6cd423', 'request_id': 'req-31b1088f-3436-4857-ab65-16ffa0b1b564'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:03.570988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5a12cc7d3e948499c57efaaeb6cd423', 'request_id': 'req-31b1088f-3436-4857-ab65-16ffa0b1b564'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:03.576660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3130ffab0b24eed8e2e3c4bff28770c', 'request_id': 'req-00ad622c-518f-4efa-b586-ebab1a3f59f1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:03.588679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3130ffab0b24eed8e2e3c4bff28770c', 'request_id': 'req-00ad622c-518f-4efa-b586-ebab1a3f59f1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:03.669200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] The token cache is being invalidate because project b5a12cc7d3e948499c57efaaeb6cd423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:31:03.669540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:03.673552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:03.676456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1077/2187] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:31:03 2022] DELETE /identity/v3/projects/b5a12cc7d3e948499c57efaaeb6cd423 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:31:03.681599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:03.684812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:03.731948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3130ffab0b24eed8e2e3c4bff28770c', 'request_id': 'req-00ad622c-518f-4efa-b586-ebab1a3f59f1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:03.754786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] The token cache is being invalidate because project b3130ffab0b24eed8e2e3c4bff28770c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:31:03.755152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:03.770239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-00ad622c-518f-4efa-b586-ebab1a3f59f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:03.777618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1111/2188] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:31:03 2022] DELETE /identity/v3/projects/b3130ffab0b24eed8e2e3c4bff28770c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:03.779800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4a202-7352-4150-bec2-9068f9960cad None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:03.780317 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4a202-7352-4150-bec2-9068f9960cad None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:03.780756 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4a202-7352-4150-bec2-9068f9960cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:03.810833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-60a4a202-7352-4150-bec2-9068f9960cad None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:03.812453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-60a4a202-7352-4150-bec2-9068f9960cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:03.824818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1088f-3436-4857-ab65-16ffa0b1b564 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:03.928112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:03.942306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:03.943076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:03.943422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:03.943766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] PATH_INFO: `/v3/users/b1592869c7804c83a1ccb6da0f2e99e0` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:03.945036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1592869c7804c83a1ccb6da0f2e99e0)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:03.950253 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:03.982165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1112/2189] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:03.984305 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4a202-7352-4150-bec2-9068f9960cad None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:03.988713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-60a4a202-7352-4150-bec2-9068f9960cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:04.021177 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4a202-7352-4150-bec2-9068f9960cad None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:04.043092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:04.044148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:04.044775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:04.045584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] PATH_INFO: `/v3/users/b1821b1a76274b27bcc1c1f4333e9480` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:04.048125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1821b1a76274b27bcc1c1f4333e9480)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:04.051823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:04.274634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1592869c7804c83a1ccb6da0f2e99e0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:04.305686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1592869c7804c83a1ccb6da0f2e99e0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:04.315555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1592869c7804c83a1ccb6da0f2e99e0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:04.341382 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1821b1a76274b27bcc1c1f4333e9480'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:04.356453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1821b1a76274b27bcc1c1f4333e9480'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:04.357239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1821b1a76274b27bcc1c1f4333e9480'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:04.371272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1821b1a76274b27bcc1c1f4333e9480'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:04.464745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1592869c7804c83a1ccb6da0f2e99e0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:04.465483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b0aa6996-fda8-4244-9069-b80c480d4bf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1592869c7804c83a1ccb6da0f2e99e0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:04.472750 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1078/2190] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:31:03 2022] DELETE /identity/v3/users/b1592869c7804c83a1ccb6da0f2e99e0 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:04.474144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b8b99-a1c3-45e3-8f14-fe3bf685740b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:04.474456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b8b99-a1c3-45e3-8f14-fe3bf685740b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:04.474731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b8b99-a1c3-45e3-8f14-fe3bf685740b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:04.504376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-cf8b8b99-a1c3-45e3-8f14-fe3bf685740b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:04.505468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cf8b8b99-a1c3-45e3-8f14-fe3bf685740b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:04.584272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1112e5a2-45f8-44c8-b14b-d4a49371171b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1821b1a76274b27bcc1c1f4333e9480'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:04.584898 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1079/2191] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:04.585877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b8b99-a1c3-45e3-8f14-fe3bf685740b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:04.596840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1113/2192] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:31:03 2022] DELETE /identity/v3/users/b1821b1a76274b27bcc1c1f4333e9480 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:04.600968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0873c3-2840-427c-bed7-26041ebac8c7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:04.601253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0873c3-2840-427c-bed7-26041ebac8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:04.601563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0873c3-2840-427c-bed7-26041ebac8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:04.604793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b8b99-a1c3-45e3-8f14-fe3bf685740b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:04.655851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-8b0873c3-2840-427c-bed7-26041ebac8c7 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:04.657514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8b0873c3-2840-427c-bed7-26041ebac8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:04.691334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef14ca8-9e7a-4523-a3b3-d6b7210efc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:04.692517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef14ca8-9e7a-4523-a3b3-d6b7210efc2a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:04.692912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef14ca8-9e7a-4523-a3b3-d6b7210efc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:04.693298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef14ca8-9e7a-4523-a3b3-d6b7210efc2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:04.694139 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef14ca8-9e7a-4523-a3b3-d6b7210efc2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:31:04.694552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef14ca8-9e7a-4523-a3b3-d6b7210efc2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:04.696300 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef14ca8-9e7a-4523-a3b3-d6b7210efc2a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:04.716361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1080/2193] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:31:04.716361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97eeaf-bb8e-45d8-9476-da7ba3058f32 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:04.716361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97eeaf-bb8e-45d8-9476-da7ba3058f32 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:04.716361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97eeaf-bb8e-45d8-9476-da7ba3058f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:04.739417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2a97eeaf-bb8e-45d8-9476-da7ba3058f32 None None] MFA Rules not processed for user `97547d56a8684009a794a42ad1c5f489`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:04.740673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2a97eeaf-bb8e-45d8-9476-da7ba3058f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:04.763976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1114/2194] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:04.769505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0873c3-2840-427c-bed7-26041ebac8c7 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:04.772467 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8b0873c3-2840-427c-bed7-26041ebac8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:04.791673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0873c3-2840-427c-bed7-26041ebac8c7 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:04.800715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04c5c8-681a-455c-9f01-3eaff5d43277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:04.801703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04c5c8-681a-455c-9f01-3eaff5d43277 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:04.802225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04c5c8-681a-455c-9f01-3eaff5d43277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:04.803566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04c5c8-681a-455c-9f01-3eaff5d43277 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:04.805518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04c5c8-681a-455c-9f01-3eaff5d43277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:04.808713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04c5c8-681a-455c-9f01-3eaff5d43277 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:04.829807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:04.833921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1081/2195] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:04.833921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97eeaf-bb8e-45d8-9476-da7ba3058f32 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:04.843082 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2a97eeaf-bb8e-45d8-9476-da7ba3058f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:04.862893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:04.926458 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1115/2196] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:31:04 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:04.928217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04c5c8-681a-455c-9f01-3eaff5d43277 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:04.950454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97eeaf-bb8e-45d8-9476-da7ba3058f32 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:05.030341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3838f-c187-4f60-ac63-b4937f73f093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:05.031754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3838f-c187-4f60-ac63-b4937f73f093 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:05.032173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3838f-c187-4f60-ac63-b4937f73f093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:05.032513 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3838f-c187-4f60-ac63-b4937f73f093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:05.048772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04c5c8-681a-455c-9f01-3eaff5d43277 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:05.050028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3838f-c187-4f60-ac63-b4937f73f093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:05.052220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3838f-c187-4f60-ac63-b4937f73f093 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:05.054240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdce46-f1a4-448d-a7ea-1c1921f4ad8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:05.054874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdce46-f1a4-448d-a7ea-1c1921f4ad8d admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:05.055300 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdce46-f1a4-448d-a7ea-1c1921f4ad8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:05.055670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdce46-f1a4-448d-a7ea-1c1921f4ad8d admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:05.056546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdce46-f1a4-448d-a7ea-1c1921f4ad8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:05.059408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdce46-f1a4-448d-a7ea-1c1921f4ad8d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:05.075645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1082/2197] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:05.255194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1116/2198] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:31:04 2022] POST /identity/v3/users => generated 287 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:05.256614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3838f-c187-4f60-ac63-b4937f73f093 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:05.282686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3838f-c187-4f60-ac63-b4937f73f093 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:05.298421 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdce46-f1a4-448d-a7ea-1c1921f4ad8d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:05.300385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-30bdce46-f1a4-448d-a7ea-1c1921f4ad8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:05.314124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408657b-5353-4208-a270-5e2b2780e3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:05.314858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408657b-5353-4208-a270-5e2b2780e3ac admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:05.315040 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408657b-5353-4208-a270-5e2b2780e3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:05.315493 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408657b-5353-4208-a270-5e2b2780e3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:05.316359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408657b-5353-4208-a270-5e2b2780e3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:05.318209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408657b-5353-4208-a270-5e2b2780e3ac admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:05.320808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdce46-f1a4-448d-a7ea-1c1921f4ad8d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:05.334017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1083/2199] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:05.338371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:05.339348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:05.339535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408657b-5353-4208-a270-5e2b2780e3ac admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:05.340831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:05.341363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] PATH_INFO: `/v3/projects/cc0c4028572f4cc299b87d3409418353` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:05.344256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc0c4028572f4cc299b87d3409418353)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:05.347479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:05.373209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408657b-5353-4208-a270-5e2b2780e3ac admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:05.391751 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6542ab-05e3-419c-9174-8a1039b03a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:05.392881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6542ab-05e3-419c-9174-8a1039b03a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:05.395619 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6542ab-05e3-419c-9174-8a1039b03a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:05.396224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6542ab-05e3-419c-9174-8a1039b03a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:05.397846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6542ab-05e3-419c-9174-8a1039b03a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:05.400274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6542ab-05e3-419c-9174-8a1039b03a9b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:05.433225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1084/2200] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:05.437459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6542ab-05e3-419c-9174-8a1039b03a9b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:05.460421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6542ab-05e3-419c-9174-8a1039b03a9b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:05.476274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52738097-7c45-43b3-8dcf-f0ce63f89a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:05.476974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52738097-7c45-43b3-8dcf-f0ce63f89a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:05.477263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52738097-7c45-43b3-8dcf-f0ce63f89a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:05.477567 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52738097-7c45-43b3-8dcf-f0ce63f89a33 admin admin] PATH_INFO: `/v3/projects/d0b889e950b644ac8a5119b28bbc2b61/users/1e06158aa43645fb834b9c87b6d84409/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:05.510378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52738097-7c45-43b3-8dcf-f0ce63f89a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b889e950b644ac8a5119b28bbc2b61, user_id=1e06158aa43645fb834b9c87b6d84409, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:05.513256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52738097-7c45-43b3-8dcf-f0ce63f89a33 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:05.592279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1085/2201] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:05 2022] PUT /identity/v3/projects/d0b889e950b644ac8a5119b28bbc2b61/users/1e06158aa43645fb834b9c87b6d84409/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:31:05.631985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc0c4028572f4cc299b87d3409418353', 'request_id': 'req-aceed918-3b42-4571-8327-a1f471d51a3c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:05.683278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc0c4028572f4cc299b87d3409418353', 'request_id': 'req-aceed918-3b42-4571-8327-a1f471d51a3c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:05.685946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df63037-34b1-4f87-9fae-6e054b96f20a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:05.686251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df63037-34b1-4f87-9fae-6e054b96f20a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:05.686540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df63037-34b1-4f87-9fae-6e054b96f20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:05.756236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc0c4028572f4cc299b87d3409418353', 'request_id': 'req-aceed918-3b42-4571-8327-a1f471d51a3c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:05.761643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2df63037-34b1-4f87-9fae-6e054b96f20a None None] MFA Rules not processed for user `1e06158aa43645fb834b9c87b6d84409`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:05.764977 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2df63037-34b1-4f87-9fae-6e054b96f20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:05.788954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] The token cache is being invalidate because project cc0c4028572f4cc299b87d3409418353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:31:05.791884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:05.795754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:05.801671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1117/2202] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:31:05 2022] DELETE /identity/v3/projects/cc0c4028572f4cc299b87d3409418353 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:05.802623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:05.807359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:05.893843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:05.918703 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceed918-3b42-4571-8327-a1f471d51a3c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:05.925119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:05.926144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:05.926458 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:05.926749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] PATH_INFO: `/v3/projects/43637c0d10a14c7cb80886bfe03273c0` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:05.927974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43637c0d10a14c7cb80886bfe03273c0)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:05.930045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:05.952197 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:05.959902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1086/2203] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:05.961433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e879b9-2ae3-4c41-a572-34a13a330a13 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:05.966200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e879b9-2ae3-4c41-a572-34a13a330a13 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:05.967069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e879b9-2ae3-4c41-a572-34a13a330a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.001271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-24e879b9-2ae3-4c41-a572-34a13a330a13 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:06.002688 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-24e879b9-2ae3-4c41-a572-34a13a330a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:06.030984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1087/2204] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:06.031421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e879b9-2ae3-4c41-a572-34a13a330a13 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:06.045220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-24e879b9-2ae3-4c41-a572-34a13a330a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:06.068377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43637c0d10a14c7cb80886bfe03273c0', 'request_id': 'req-f2239cae-3bea-41bc-ab47-9cdae3b633f9'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:06.090222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43637c0d10a14c7cb80886bfe03273c0', 'request_id': 'req-f2239cae-3bea-41bc-ab47-9cdae3b633f9'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:06.169356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e879b9-2ae3-4c41-a572-34a13a330a13 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:06.183423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43637c0d10a14c7cb80886bfe03273c0', 'request_id': 'req-f2239cae-3bea-41bc-ab47-9cdae3b633f9'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:06.206264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] The token cache is being invalidate because project 43637c0d10a14c7cb80886bfe03273c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:31:06.206565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:06.208541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f2239cae-3bea-41bc-ab47-9cdae3b633f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:06.210728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1118/2205] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:31:05 2022] DELETE /identity/v3/projects/43637c0d10a14c7cb80886bfe03273c0 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:06.218780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e307247-b63c-4a1d-a010-31939592e997 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:06.219077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e307247-b63c-4a1d-a010-31939592e997 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:06.219366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e307247-b63c-4a1d-a010-31939592e997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.250654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145875a-7ca1-44df-ae2a-08f006eb0360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:06.251613 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145875a-7ca1-44df-ae2a-08f006eb0360 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:06.253778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145875a-7ca1-44df-ae2a-08f006eb0360 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:06.254093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4e307247-b63c-4a1d-a010-31939592e997 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:06.254268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145875a-7ca1-44df-ae2a-08f006eb0360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.255868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4e307247-b63c-4a1d-a010-31939592e997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:06.257400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d145875a-7ca1-44df-ae2a-08f006eb0360 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:06.401334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:06.427746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1119/2206] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:06.429997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928ed5-a338-4c89-977b-445152406f46 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:06.430291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928ed5-a338-4c89-977b-445152406f46 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:06.432114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928ed5-a338-4c89-977b-445152406f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.454758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145875a-7ca1-44df-ae2a-08f006eb0360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:06.460142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145875a-7ca1-44df-ae2a-08f006eb0360 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:06.474849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1088/2207] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:06.475964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145875a-7ca1-44df-ae2a-08f006eb0360 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:06.489997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145875a-7ca1-44df-ae2a-08f006eb0360 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:06.496757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af756f2-b880-469b-a495-fd8111e93fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:06.498148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af756f2-b880-469b-a495-fd8111e93fce admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:06.501297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af756f2-b880-469b-a495-fd8111e93fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:06.501297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af756f2-b880-469b-a495-fd8111e93fce admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.501297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af756f2-b880-469b-a495-fd8111e93fce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:31:06.501297 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af756f2-b880-469b-a495-fd8111e93fce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:06.504115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af756f2-b880-469b-a495-fd8111e93fce admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:06.508514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-78928ed5-a338-4c89-977b-445152406f46 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:06.510697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-78928ed5-a338-4c89-977b-445152406f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:06.536149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1089/2208] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:31:06.544621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af756f2-b880-469b-a495-fd8111e93fce admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:06.563227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af756f2-b880-469b-a495-fd8111e93fce admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:06.567033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1120/2209] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:06.569412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18dcdf-b3d6-427e-88e6-dd63facb8d6a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:06.572249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18dcdf-b3d6-427e-88e6-dd63facb8d6a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:06.572249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18dcdf-b3d6-427e-88e6-dd63facb8d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.574030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644f6d9-172f-4394-b795-fc1ab9ae93f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:06.575019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644f6d9-172f-4394-b795-fc1ab9ae93f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:06.575509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644f6d9-172f-4394-b795-fc1ab9ae93f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:06.576008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644f6d9-172f-4394-b795-fc1ab9ae93f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.577583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644f6d9-172f-4394-b795-fc1ab9ae93f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:06.580412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644f6d9-172f-4394-b795-fc1ab9ae93f4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:06.582032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=83,keystone:INSERT=4,keystone:DELETE=13 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:06.627794 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-8a18dcdf-b3d6-427e-88e6-dd63facb8d6a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:06.629059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8a18dcdf-b3d6-427e-88e6-dd63facb8d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:06.642606 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1121/2210] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:06.643525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18dcdf-b3d6-427e-88e6-dd63facb8d6a None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:06.659821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18dcdf-b3d6-427e-88e6-dd63facb8d6a None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:06.670182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26204615-9379-4d8d-966e-c3dff1e511ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:06.671280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26204615-9379-4d8d-966e-c3dff1e511ab admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:06.672598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26204615-9379-4d8d-966e-c3dff1e511ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:06.674070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26204615-9379-4d8d-966e-c3dff1e511ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.674987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26204615-9379-4d8d-966e-c3dff1e511ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:31:06.675478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26204615-9379-4d8d-966e-c3dff1e511ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:06.678144 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26204615-9379-4d8d-966e-c3dff1e511ab admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:06.709350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1122/2211] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:31:06.714578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26204615-9379-4d8d-966e-c3dff1e511ab admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:06.719614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1090/2212] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:31:06 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:31:06.720673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644f6d9-172f-4394-b795-fc1ab9ae93f4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:06.735793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26204615-9379-4d8d-966e-c3dff1e511ab admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:06.835122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644f6d9-172f-4394-b795-fc1ab9ae93f4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:06.843448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31bb13-de5d-476c-90bd-fd37e0af33a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:06.844326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31bb13-de5d-476c-90bd-fd37e0af33a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:06.844762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31bb13-de5d-476c-90bd-fd37e0af33a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:06.845347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31bb13-de5d-476c-90bd-fd37e0af33a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.846240 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31bb13-de5d-476c-90bd-fd37e0af33a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:06.853797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31bb13-de5d-476c-90bd-fd37e0af33a3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:06.854207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72fc9e-5bcf-4216-8e24-691452eb1503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:06.854207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72fc9e-5bcf-4216-8e24-691452eb1503 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:06.854207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72fc9e-5bcf-4216-8e24-691452eb1503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:06.854207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72fc9e-5bcf-4216-8e24-691452eb1503 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.854604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72fc9e-5bcf-4216-8e24-691452eb1503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:06.854604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72fc9e-5bcf-4216-8e24-691452eb1503 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:06.942845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1091/2213] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:31:06 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:06.947308 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31bb13-de5d-476c-90bd-fd37e0af33a3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:06.973611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31bb13-de5d-476c-90bd-fd37e0af33a3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:06.984734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40570175-a5c6-4559-a632-51c78002a820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:06.985427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40570175-a5c6-4559-a632-51c78002a820 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:06.985707 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40570175-a5c6-4559-a632-51c78002a820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:06.985987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40570175-a5c6-4559-a632-51c78002a820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:06.986707 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40570175-a5c6-4559-a632-51c78002a820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:06.995184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40570175-a5c6-4559-a632-51c78002a820 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:07.020156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1123/2214] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:31:06 2022] POST /identity/v3/projects => generated 377 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:07.026532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1092/2215] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:07.032643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40570175-a5c6-4559-a632-51c78002a820 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:07.032853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72fc9e-5bcf-4216-8e24-691452eb1503 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:07.044156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40570175-a5c6-4559-a632-51c78002a820 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:07.098519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:07.150222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190a21d-e28c-4aa1-9886-b516edc1aa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:07.151123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190a21d-e28c-4aa1-9886-b516edc1aa2a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:07.151598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190a21d-e28c-4aa1-9886-b516edc1aa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:07.152075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190a21d-e28c-4aa1-9886-b516edc1aa2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:07.152797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190a21d-e28c-4aa1-9886-b516edc1aa2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:07.154480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190a21d-e28c-4aa1-9886-b516edc1aa2a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:07.166607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1093/2216] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:07.170538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190a21d-e28c-4aa1-9886-b516edc1aa2a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:07.179038 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72fc9e-5bcf-4216-8e24-691452eb1503 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:07.185693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190a21d-e28c-4aa1-9886-b516edc1aa2a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:07.186516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a29731-ffe3-406e-91cc-272bab439e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:07.187474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a29731-ffe3-406e-91cc-272bab439e9c admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:07.187973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a29731-ffe3-406e-91cc-272bab439e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:07.188363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a29731-ffe3-406e-91cc-272bab439e9c admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:07.189597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a29731-ffe3-406e-91cc-272bab439e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:07.190984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517bdd5-f5c3-40a6-9460-4d91ab0693b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:07.191715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517bdd5-f5c3-40a6-9460-4d91ab0693b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:07.192015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517bdd5-f5c3-40a6-9460-4d91ab0693b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:07.192331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517bdd5-f5c3-40a6-9460-4d91ab0693b6 admin admin] PATH_INFO: `/v3/projects/782e41a60a084375bbb1758b7516aa82/users/eaff9a3a21df473fa51185af7c154f3e/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:07.193077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a29731-ffe3-406e-91cc-272bab439e9c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:07.215359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517bdd5-f5c3-40a6-9460-4d91ab0693b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=782e41a60a084375bbb1758b7516aa82, user_id=eaff9a3a21df473fa51185af7c154f3e, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:07.219371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517bdd5-f5c3-40a6-9460-4d91ab0693b6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:07.323340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1124/2217] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:31:07 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:07.327224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a29731-ffe3-406e-91cc-272bab439e9c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:07.343147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a29731-ffe3-406e-91cc-272bab439e9c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:07.350575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33df5b4-c2b0-4800-a4d9-b3def8a4a30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:07.351221 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33df5b4-c2b0-4800-a4d9-b3def8a4a30b admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:07.359638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33df5b4-c2b0-4800-a4d9-b3def8a4a30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:07.359638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33df5b4-c2b0-4800-a4d9-b3def8a4a30b admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:07.359638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33df5b4-c2b0-4800-a4d9-b3def8a4a30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:07.359638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33df5b4-c2b0-4800-a4d9-b3def8a4a30b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:07.372831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1125/2218] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:31:07.373954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33df5b4-c2b0-4800-a4d9-b3def8a4a30b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:07.394856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33df5b4-c2b0-4800-a4d9-b3def8a4a30b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:07.397635 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1094/2219] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:07 2022] PUT /identity/v3/projects/782e41a60a084375bbb1758b7516aa82/users/eaff9a3a21df473fa51185af7c154f3e/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:31:07.411359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1cd4-2451-4bc2-92f6-7517f48f7b8d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:07.411359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1cd4-2451-4bc2-92f6-7517f48f7b8d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:07.412027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1cd4-2451-4bc2-92f6-7517f48f7b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:07.473296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-345f1cd4-2451-4bc2-92f6-7517f48f7b8d None None] MFA Rules not processed for user `eaff9a3a21df473fa51185af7c154f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:07.475279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-345f1cd4-2451-4bc2-92f6-7517f48f7b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:07.520952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace48a71-2244-4eaa-b687-7606a0eab822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:07.522303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48a71-2244-4eaa-b687-7606a0eab822 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:07.522776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48a71-2244-4eaa-b687-7606a0eab822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:07.523212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48a71-2244-4eaa-b687-7606a0eab822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:07.524081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace48a71-2244-4eaa-b687-7606a0eab822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:07.526764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace48a71-2244-4eaa-b687-7606a0eab822 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:07.544562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1126/2220] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:31:07.545814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace48a71-2244-4eaa-b687-7606a0eab822 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:07.558666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace48a71-2244-4eaa-b687-7606a0eab822 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:07.572491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcffba-a57e-4bb9-ba05-41af03acaa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:07.573593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcffba-a57e-4bb9-ba05-41af03acaa00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:07.574586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcffba-a57e-4bb9-ba05-41af03acaa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:07.575030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcffba-a57e-4bb9-ba05-41af03acaa00 admin admin] PATH_INFO: `/v3/projects/47b406d9a12c461a84aa0331d2890990/users/b1868c7d0ec8443b9ccb698ee02928b6/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:07.589796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1095/2221] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:07.591740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f1cd4-2451-4bc2-92f6-7517f48f7b8d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:07.595178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcffba-a57e-4bb9-ba05-41af03acaa00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47b406d9a12c461a84aa0331d2890990, user_id=b1868c7d0ec8443b9ccb698ee02928b6, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:07.599149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcffba-a57e-4bb9-ba05-41af03acaa00 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:07.635850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f1cd4-2451-4bc2-92f6-7517f48f7b8d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:07.643066 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c45be-00a3-458c-b85c-0e78a78a1e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:07.643795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c45be-00a3-458c-b85c-0e78a78a1e82 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:07.644188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c45be-00a3-458c-b85c-0e78a78a1e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:07.645816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c45be-00a3-458c-b85c-0e78a78a1e82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:07.658403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c45be-00a3-458c-b85c-0e78a78a1e82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:07.658943 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:07.661462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c45be-00a3-458c-b85c-0e78a78a1e82 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:07.711822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1127/2222] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:07 2022] PUT /identity/v3/projects/47b406d9a12c461a84aa0331d2890990/users/b1868c7d0ec8443b9ccb698ee02928b6/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:07.719595 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7622fd7-56a1-41dc-bf2d-ab7f80a46764 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:07.719896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7622fd7-56a1-41dc-bf2d-ab7f80a46764 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:07.720181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7622fd7-56a1-41dc-bf2d-ab7f80a46764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:07.758652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f7622fd7-56a1-41dc-bf2d-ab7f80a46764 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:07.765610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f7622fd7-56a1-41dc-bf2d-ab7f80a46764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:07.826689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1096/2223] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:31:07 2022] POST /identity/v3/projects => generated 385 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:07.828007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c45be-00a3-458c-b85c-0e78a78a1e82 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:07.848892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:07.881084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1128/2224] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:07.891755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7622fd7-56a1-41dc-bf2d-ab7f80a46764 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:07.893234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f7622fd7-56a1-41dc-bf2d-ab7f80a46764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:07.967505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c45be-00a3-458c-b85c-0e78a78a1e82 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:07.979545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7993537-32ed-4264-b750-dc93e4167432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:07.981747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7993537-32ed-4264-b750-dc93e4167432 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:07.982381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7993537-32ed-4264-b750-dc93e4167432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:07.984533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7993537-32ed-4264-b750-dc93e4167432 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:08.006757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7993537-32ed-4264-b750-dc93e4167432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:08.014650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7993537-32ed-4264-b750-dc93e4167432 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:08.025699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7622fd7-56a1-41dc-bf2d-ab7f80a46764 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:08.261888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab088a4b-25c7-49bf-9349-e30c5333a66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:08.261888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab088a4b-25c7-49bf-9349-e30c5333a66e service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:08.261888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab088a4b-25c7-49bf-9349-e30c5333a66e service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:08.261888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab088a4b-25c7-49bf-9349-e30c5333a66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:08.263166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab088a4b-25c7-49bf-9349-e30c5333a66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:08.263166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab088a4b-25c7-49bf-9349-e30c5333a66e service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:08.263166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1129/2225] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:31:08.939207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1097/2226] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:31:07 2022] POST /identity/v3/users => generated 299 bytes in 1112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:08.944114 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab088a4b-25c7-49bf-9349-e30c5333a66e service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:08.955793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab088a4b-25c7-49bf-9349-e30c5333a66e service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:08.960034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bedac-e9af-4eea-99f3-26ae13a9ef60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:08.960714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bedac-e9af-4eea-99f3-26ae13a9ef60 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:08.961001 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bedac-e9af-4eea-99f3-26ae13a9ef60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:08.961276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bedac-e9af-4eea-99f3-26ae13a9ef60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:08.962047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bedac-e9af-4eea-99f3-26ae13a9ef60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:08.964623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bedac-e9af-4eea-99f3-26ae13a9ef60 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:08.976447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1130/2227] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:31:08.982034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7993537-32ed-4264-b750-dc93e4167432 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:08.994493 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7993537-32ed-4264-b750-dc93e4167432 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:09.000079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb520cc-e957-45a0-9377-cdb9a368c399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:09.000933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb520cc-e957-45a0-9377-cdb9a368c399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:09.001290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb520cc-e957-45a0-9377-cdb9a368c399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:09.001709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb520cc-e957-45a0-9377-cdb9a368c399 admin admin] PATH_INFO: `/v3/projects/591f16315ac342e896bb67786dec76aa/users/e384a95dc9c8438ab7befbddd7bdec4c/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:09.041371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb520cc-e957-45a0-9377-cdb9a368c399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591f16315ac342e896bb67786dec76aa, user_id=e384a95dc9c8438ab7befbddd7bdec4c, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:09.044334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb520cc-e957-45a0-9377-cdb9a368c399 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:09.184171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1098/2228] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:08 2022] PUT /identity/v3/projects/591f16315ac342e896bb67786dec76aa/users/e384a95dc9c8438ab7befbddd7bdec4c/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:09.193614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bedac-e9af-4eea-99f3-26ae13a9ef60 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:09.199151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bedac-e9af-4eea-99f3-26ae13a9ef60 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:09.264841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e2356-86b3-4cc3-a11f-34f634f11f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:09.265629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e2356-86b3-4cc3-a11f-34f634f11f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:09.265992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e2356-86b3-4cc3-a11f-34f634f11f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:09.266344 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e2356-86b3-4cc3-a11f-34f634f11f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:09.268694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e2356-86b3-4cc3-a11f-34f634f11f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:09.269129 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e2356-86b3-4cc3-a11f-34f634f11f93 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:09.284204 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1131/2229] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:31:09.288526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb520cc-e957-45a0-9377-cdb9a368c399 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:09.300578 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb520cc-e957-45a0-9377-cdb9a368c399 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:09.305614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dc2cc-2e70-4cfb-a2ef-339f51c0ca07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:09.306552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dc2cc-2e70-4cfb-a2ef-339f51c0ca07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:09.306850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dc2cc-2e70-4cfb-a2ef-339f51c0ca07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:09.307135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dc2cc-2e70-4cfb-a2ef-339f51c0ca07 admin admin] PATH_INFO: `/v3/projects/591f16315ac342e896bb67786dec76aa/users/e384a95dc9c8438ab7befbddd7bdec4c/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:09.311867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3dc2cc-2e70-4cfb-a2ef-339f51c0ca07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591f16315ac342e896bb67786dec76aa, user_id=e384a95dc9c8438ab7befbddd7bdec4c, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:09.311993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3dc2cc-2e70-4cfb-a2ef-339f51c0ca07 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:09.323218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1099/2230] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:09 2022] PUT /identity/v3/projects/591f16315ac342e896bb67786dec76aa/users/e384a95dc9c8438ab7befbddd7bdec4c/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:09.398571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc28cb-8ebb-4889-83f8-cabf212b067d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:09.398935 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc28cb-8ebb-4889-83f8-cabf212b067d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:09.399177 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc28cb-8ebb-4889-83f8-cabf212b067d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:09.469269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-a5dc28cb-8ebb-4889-83f8-cabf212b067d None None] MFA Rules not processed for user `e384a95dc9c8438ab7befbddd7bdec4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:09.472165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a5dc28cb-8ebb-4889-83f8-cabf212b067d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:09.601136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1132/2231] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:09.613568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dc2cc-2e70-4cfb-a2ef-339f51c0ca07 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:09.615739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7b3dc2cc-2e70-4cfb-a2ef-339f51c0ca07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:09.770050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dc2cc-2e70-4cfb-a2ef-339f51c0ca07 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:09.775441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0caec-c02b-4880-aded-3b2fecd026ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:09.776929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0caec-c02b-4880-aded-3b2fecd026ec service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:09.777471 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0caec-c02b-4880-aded-3b2fecd026ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:09.777932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0caec-c02b-4880-aded-3b2fecd026ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:09.816639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0caec-c02b-4880-aded-3b2fecd026ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:09.819965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0caec-c02b-4880-aded-3b2fecd026ec service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:09.865491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1100/2232] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:10.000268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:10.687023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:11.834911 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc28cb-8ebb-4889-83f8-cabf212b067d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:11.842327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:11.966848 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc28cb-8ebb-4889-83f8-cabf212b067d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:11.973056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:11.974269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:11.974774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:11.975229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] PATH_INFO: `/v3/users/eaff9a3a21df473fa51185af7c154f3e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:11.977524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaff9a3a21df473fa51185af7c154f3e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:11.980547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:12.184594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaff9a3a21df473fa51185af7c154f3e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:12.247214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaff9a3a21df473fa51185af7c154f3e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:12.247351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaff9a3a21df473fa51185af7c154f3e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:12.279094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaff9a3a21df473fa51185af7c154f3e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:12.380521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaff9a3a21df473fa51185af7c154f3e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:12.388062 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1133/2233] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:31:11 2022] DELETE /identity/v3/users/eaff9a3a21df473fa51185af7c154f3e => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:12.396489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86fafe-1aae-4052-bf9a-9862d1ddbfb8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:12.396985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86fafe-1aae-4052-bf9a-9862d1ddbfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:12.397259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86fafe-1aae-4052-bf9a-9862d1ddbfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:12.424964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2c86fafe-1aae-4052-bf9a-9862d1ddbfb8 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:12.426609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2c86fafe-1aae-4052-bf9a-9862d1ddbfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:12.545748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1101/2234] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:12.573422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:12.576013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:12.723478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27b770-d0da-425c-bea9-3d90783c3212 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:12.728342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac6cc2-688f-44aa-9e48-6e52346786ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:12.729053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac6cc2-688f-44aa-9e48-6e52346786ca service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:12.729598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac6cc2-688f-44aa-9e48-6e52346786ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:12.729901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac6cc2-688f-44aa-9e48-6e52346786ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:12.742203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac6cc2-688f-44aa-9e48-6e52346786ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:12.744460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac6cc2-688f-44aa-9e48-6e52346786ca service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:12.772608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1134/2235] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:13.972844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86fafe-1aae-4052-bf9a-9862d1ddbfb8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:13.974458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2c86fafe-1aae-4052-bf9a-9862d1ddbfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:14.015393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac6cc2-688f-44aa-9e48-6e52346786ca service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:14.018302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-62ac6cc2-688f-44aa-9e48-6e52346786ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:14.061999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac6cc2-688f-44aa-9e48-6e52346786ca service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:14.084382 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:14.086156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:14.098923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:14.098923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] PATH_INFO: `/v3/projects/782e41a60a084375bbb1758b7516aa82` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:14.107594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] RBAC: Authorizing `identity:delete_project(project_id=782e41a60a084375bbb1758b7516aa82)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:14.112488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:14.208133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:14.357455 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86fafe-1aae-4052-bf9a-9862d1ddbfb8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:14.358606 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '782e41a60a084375bbb1758b7516aa82', 'request_id': 'req-ed3e1f97-873b-426d-ba90-bcd007cc455b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:14.379472 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '782e41a60a084375bbb1758b7516aa82', 'request_id': 'req-ed3e1f97-873b-426d-ba90-bcd007cc455b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:14.527087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '782e41a60a084375bbb1758b7516aa82', 'request_id': 'req-ed3e1f97-873b-426d-ba90-bcd007cc455b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:14.560514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] The token cache is being invalidate because project 782e41a60a084375bbb1758b7516aa82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:31:14.560819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:14.563073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:14.565759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908895e-72a2-4a49-a346-08c908afe173 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:14.567105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1135/2236] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:31:14 2022] DELETE /identity/v3/projects/782e41a60a084375bbb1758b7516aa82 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:31:14.570279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908895e-72a2-4a49-a346-08c908afe173 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:14.570279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908895e-72a2-4a49-a346-08c908afe173 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:14.570279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908895e-72a2-4a49-a346-08c908afe173 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:14.571235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:14.578637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d908895e-72a2-4a49-a346-08c908afe173 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:14.587847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:14.608370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908895e-72a2-4a49-a346-08c908afe173 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:14.609853 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908895e-72a2-4a49-a346-08c908afe173 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:14.641463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1102/2237] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:14.841940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e1f97-873b-426d-ba90-bcd007cc455b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:14.862757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:14.863441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:14.864008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:14.864297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] PATH_INFO: `/v3/users/e384a95dc9c8438ab7befbddd7bdec4c` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:14.872689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e384a95dc9c8438ab7befbddd7bdec4c)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:14.879228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:15.225265 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e384a95dc9c8438ab7befbddd7bdec4c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:15.251302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e384a95dc9c8438ab7befbddd7bdec4c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:15.252083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e384a95dc9c8438ab7befbddd7bdec4c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:15.280808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e384a95dc9c8438ab7befbddd7bdec4c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:15.390013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-47fe482d-c778-45d7-bf61-be88a48be899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e384a95dc9c8438ab7befbddd7bdec4c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:15.410303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1136/2238] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:31:14 2022] DELETE /identity/v3/users/e384a95dc9c8438ab7befbddd7bdec4c => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:16.975848 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908895e-72a2-4a49-a346-08c908afe173 service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:16.995922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908895e-72a2-4a49-a346-08c908afe173 service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:17.091356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:17.158136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:17.158824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:17.159118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:17.159400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] PATH_INFO: `/v3/projects/591f16315ac342e896bb67786dec76aa` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:17.163557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=591f16315ac342e896bb67786dec76aa)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:17.169839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:17.432459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591f16315ac342e896bb67786dec76aa', 'request_id': 'req-8f53fcfe-4324-43ef-99e1-9a438f4114ec'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:17.463649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591f16315ac342e896bb67786dec76aa', 'request_id': 'req-8f53fcfe-4324-43ef-99e1-9a438f4114ec'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:17.492645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591f16315ac342e896bb67786dec76aa', 'request_id': 'req-8f53fcfe-4324-43ef-99e1-9a438f4114ec'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:17.598685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] The token cache is being invalidate because project 591f16315ac342e896bb67786dec76aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:31:17.599381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:17.600507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8f53fcfe-4324-43ef-99e1-9a438f4114ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:17.603347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1103/2239] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:31:16 2022] DELETE /identity/v3/projects/591f16315ac342e896bb67786dec76aa => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:17.608045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ff65f-9590-41c9-ad7b-0b8ea6636146 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:17.612956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ff65f-9590-41c9-ad7b-0b8ea6636146 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:17.612956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ff65f-9590-41c9-ad7b-0b8ea6636146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:17.675505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b11ff65f-9590-41c9-ad7b-0b8ea6636146 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:17.677862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b11ff65f-9590-41c9-ad7b-0b8ea6636146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:17.759959 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:17.881954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88176bfc-d966-49b6-8e7b-e8006b41032c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:17.882396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88176bfc-d966-49b6-8e7b-e8006b41032c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:17.882811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88176bfc-d966-49b6-8e7b-e8006b41032c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:17.891219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1137/2240] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:17.967506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-88176bfc-d966-49b6-8e7b-e8006b41032c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:17.976133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-88176bfc-d966-49b6-8e7b-e8006b41032c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:18.001743 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ff65f-9590-41c9-ad7b-0b8ea6636146 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:18.007607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1104/2241] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:18.016192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ff65f-9590-41c9-ad7b-0b8ea6636146 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:18.022196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0226c1-19c5-431c-834f-1540b23d0b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:18.023135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0226c1-19c5-431c-834f-1540b23d0b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:18.023554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0226c1-19c5-431c-834f-1540b23d0b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:18.023986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0226c1-19c5-431c-834f-1540b23d0b7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:18.025666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0226c1-19c5-431c-834f-1540b23d0b7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:31:18.026213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0226c1-19c5-431c-834f-1540b23d0b7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:18.029164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0226c1-19c5-431c-834f-1540b23d0b7b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:18.065164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1138/2242] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:31:18.071824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88176bfc-d966-49b6-8e7b-e8006b41032c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:18.086665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88176bfc-d966-49b6-8e7b-e8006b41032c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:18.101259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b1edd-0ad9-4da6-bc60-686a0cf8dfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:18.105770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b1edd-0ad9-4da6-bc60-686a0cf8dfe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:18.105770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b1edd-0ad9-4da6-bc60-686a0cf8dfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:18.105770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b1edd-0ad9-4da6-bc60-686a0cf8dfe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:18.106831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b1edd-0ad9-4da6-bc60-686a0cf8dfe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:18.116443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b1edd-0ad9-4da6-bc60-686a0cf8dfe1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:18.308880 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1105/2243] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:31:18 2022] POST /identity/v3/projects => generated 371 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:18.313188 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0226c1-19c5-431c-834f-1540b23d0b7b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:18.417453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0226c1-19c5-431c-834f-1540b23d0b7b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:18.625638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d413-921d-40ba-9cca-adcd6a138e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:18.626858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d413-921d-40ba-9cca-adcd6a138e85 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:18.627389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d413-921d-40ba-9cca-adcd6a138e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:18.631235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d413-921d-40ba-9cca-adcd6a138e85 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:18.632269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219d413-921d-40ba-9cca-adcd6a138e85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:18.641675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219d413-921d-40ba-9cca-adcd6a138e85 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:18.798119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1139/2244] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:31:18 2022] POST /identity/v3/users => generated 286 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:18.802525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b1edd-0ad9-4da6-bc60-686a0cf8dfe1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:18.819243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b1edd-0ad9-4da6-bc60-686a0cf8dfe1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:18.834812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe124d18-ba8f-47d6-88af-d8cabe51ab9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:18.837876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe124d18-ba8f-47d6-88af-d8cabe51ab9a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:18.838453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe124d18-ba8f-47d6-88af-d8cabe51ab9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:18.839118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe124d18-ba8f-47d6-88af-d8cabe51ab9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:18.842693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe124d18-ba8f-47d6-88af-d8cabe51ab9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:18.846762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe124d18-ba8f-47d6-88af-d8cabe51ab9a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:18.868697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1106/2245] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:18.872694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d413-921d-40ba-9cca-adcd6a138e85 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:18.889974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d413-921d-40ba-9cca-adcd6a138e85 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:18.903329 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f224b6-665b-4f27-bb3b-260b45260284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:18.904381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f224b6-665b-4f27-bb3b-260b45260284 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:18.904889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f224b6-665b-4f27-bb3b-260b45260284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:18.905475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f224b6-665b-4f27-bb3b-260b45260284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:18.907048 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f224b6-665b-4f27-bb3b-260b45260284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:18.914464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f224b6-665b-4f27-bb3b-260b45260284 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:18.941503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1140/2246] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:18.945425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe124d18-ba8f-47d6-88af-d8cabe51ab9a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:18.961004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe124d18-ba8f-47d6-88af-d8cabe51ab9a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:18.972560 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f735a-0af4-4762-9136-c56d4e01f452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:18.976662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f735a-0af4-4762-9136-c56d4e01f452 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:18.977289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f735a-0af4-4762-9136-c56d4e01f452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:18.977517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f735a-0af4-4762-9136-c56d4e01f452 admin admin] PATH_INFO: `/v3/projects/6f42ddc1f920465f88baa24820a3976f/users/061889c63b65453f8c0d409acfab420e/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:19.020829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f735a-0af4-4762-9136-c56d4e01f452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f42ddc1f920465f88baa24820a3976f, user_id=061889c63b65453f8c0d409acfab420e, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:19.024054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f735a-0af4-4762-9136-c56d4e01f452 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:19.125710 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1107/2247] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:18 2022] PUT /identity/v3/projects/6f42ddc1f920465f88baa24820a3976f/users/061889c63b65453f8c0d409acfab420e/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:19.133526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab998031-eff7-49de-a9a6-00cb450c5902 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:19.134868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab998031-eff7-49de-a9a6-00cb450c5902 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:19.135799 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab998031-eff7-49de-a9a6-00cb450c5902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:19.179909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-ab998031-eff7-49de-a9a6-00cb450c5902 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:19.182624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ab998031-eff7-49de-a9a6-00cb450c5902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:19.356910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1141/2248] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:19.509565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f735a-0af4-4762-9136-c56d4e01f452 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:19.512032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0c2f735a-0af4-4762-9136-c56d4e01f452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:19.725574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f735a-0af4-4762-9136-c56d4e01f452 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:19.748341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2912c0-40c1-4c97-9286-e44f784a99b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:19.750340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2912c0-40c1-4c97-9286-e44f784a99b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:19.750639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2912c0-40c1-4c97-9286-e44f784a99b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:19.750920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2912c0-40c1-4c97-9286-e44f784a99b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:19.786954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2912c0-40c1-4c97-9286-e44f784a99b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:19.788432 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2912c0-40c1-4c97-9286-e44f784a99b6 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:19.824482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1108/2249] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:23.469694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8a100-77cb-46b1-b015-526042a2c1af None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:23.469694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8a100-77cb-46b1-b015-526042a2c1af None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:23.469694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8a100-77cb-46b1-b015-526042a2c1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:23.509515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-67d8a100-77cb-46b1-b015-526042a2c1af None None] MFA Rules not processed for user `b1868c7d0ec8443b9ccb698ee02928b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:23.510808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-67d8a100-77cb-46b1-b015-526042a2c1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:23.553830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:23.613683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1142/2250] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:23.615244 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2912c0-40c1-4c97-9286-e44f784a99b6 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:23.616801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9c2912c0-40c1-4c97-9286-e44f784a99b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:23.638331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:23.647277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2912c0-40c1-4c97-9286-e44f784a99b6 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:23.654995 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29705ca2-42f7-43cf-803d-df0facbb6934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:23.655949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29705ca2-42f7-43cf-803d-df0facbb6934 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:23.656234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29705ca2-42f7-43cf-803d-df0facbb6934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:23.656673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29705ca2-42f7-43cf-803d-df0facbb6934 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:23.657411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29705ca2-42f7-43cf-803d-df0facbb6934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:23.659469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29705ca2-42f7-43cf-803d-df0facbb6934 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:23.825999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ab96c-3942-47f9-a8b8-292485290c8e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:23.826407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ab96c-3942-47f9-a8b8-292485290c8e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:23.826820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ab96c-3942-47f9-a8b8-292485290c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:23.832765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1109/2251] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:31:23 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:23.833760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29705ca2-42f7-43cf-803d-df0facbb6934 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:23.878750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-6b4ab96c-3942-47f9-a8b8-292485290c8e None None] MFA Rules not processed for user `061889c63b65453f8c0d409acfab420e`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:23.888973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6b4ab96c-3942-47f9-a8b8-292485290c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:24.017919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29705ca2-42f7-43cf-803d-df0facbb6934 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:24.023768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dc50d-bb3c-4f54-a1d7-5be024cf3ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:24.025206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc50d-bb3c-4f54-a1d7-5be024cf3ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.025671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc50d-bb3c-4f54-a1d7-5be024cf3ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.026055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc50d-bb3c-4f54-a1d7-5be024cf3ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.027025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dc50d-bb3c-4f54-a1d7-5be024cf3ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:24.030166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dc50d-bb3c-4f54-a1d7-5be024cf3ea6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:24.038396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1143/2252] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:31:24.042990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ab96c-3942-47f9-a8b8-292485290c8e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:24.058477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ab96c-3942-47f9-a8b8-292485290c8e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:24.066233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7958337-e23b-4b8b-a18d-1bbe77ccb9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:24.066233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7958337-e23b-4b8b-a18d-1bbe77ccb9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.066233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7958337-e23b-4b8b-a18d-1bbe77ccb9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.066233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7958337-e23b-4b8b-a18d-1bbe77ccb9a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.066674 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7958337-e23b-4b8b-a18d-1bbe77ccb9a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:24.067578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7958337-e23b-4b8b-a18d-1bbe77ccb9a7 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:24.155772 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1110/2253] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:31:23 2022] POST /identity/v3/users => generated 295 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:24.156824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dc50d-bb3c-4f54-a1d7-5be024cf3ea6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:24.170604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dc50d-bb3c-4f54-a1d7-5be024cf3ea6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:24.176017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777facc-c334-41b5-b781-7248a91a3990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:24.176674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777facc-c334-41b5-b781-7248a91a3990 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.176966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777facc-c334-41b5-b781-7248a91a3990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.177340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777facc-c334-41b5-b781-7248a91a3990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.178106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777facc-c334-41b5-b781-7248a91a3990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:24.179822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777facc-c334-41b5-b781-7248a91a3990 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:24.195790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1111/2254] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:24.197801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777facc-c334-41b5-b781-7248a91a3990 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:24.207533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1144/2255] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:31:24 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:31:24.211080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777facc-c334-41b5-b781-7248a91a3990 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:24.216925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7958337-e23b-4b8b-a18d-1bbe77ccb9a7 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:24.301701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a18d25-d16d-4ee2-a705-3a0b896a420c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:24.306359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a18d25-d16d-4ee2-a705-3a0b896a420c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.306359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a18d25-d16d-4ee2-a705-3a0b896a420c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.306359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a18d25-d16d-4ee2-a705-3a0b896a420c admin admin] PATH_INFO: `/v3/projects/940cf692c0554607a4d43de7c23a22a2/users/b28a7273be4947a18f690277ca61575e/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.323247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a18d25-d16d-4ee2-a705-3a0b896a420c admin admin] RBAC: Authorizing `identity:create_grant(project_id=940cf692c0554607a4d43de7c23a22a2, user_id=b28a7273be4947a18f690277ca61575e, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:24.326465 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a18d25-d16d-4ee2-a705-3a0b896a420c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:24.370259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7958337-e23b-4b8b-a18d-1bbe77ccb9a7 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:24.390432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a12fa-acae-4a54-b902-3039a14e0763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:24.391363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a12fa-acae-4a54-b902-3039a14e0763 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.391864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a12fa-acae-4a54-b902-3039a14e0763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.392305 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a12fa-acae-4a54-b902-3039a14e0763 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.393193 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a12fa-acae-4a54-b902-3039a14e0763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:24.395528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a12fa-acae-4a54-b902-3039a14e0763 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:24.449051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1112/2256] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:24 2022] PUT /identity/v3/projects/940cf692c0554607a4d43de7c23a22a2/users/b28a7273be4947a18f690277ca61575e/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:24.450112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a18d25-d16d-4ee2-a705-3a0b896a420c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:24.453131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f4a18d25-d16d-4ee2-a705-3a0b896a420c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:24.567044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a18d25-d16d-4ee2-a705-3a0b896a420c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:24.574706 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:24.575640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.576076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.576536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] PATH_INFO: `/v3/users/1e06158aa43645fb834b9c87b6d84409` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.578188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e06158aa43645fb834b9c87b6d84409)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:24.580675 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:24.584304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:24.620887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1145/2257] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:31:24 2022] POST /identity/v3/users => generated 293 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:24.621790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a12fa-acae-4a54-b902-3039a14e0763 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:24.648135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a12fa-acae-4a54-b902-3039a14e0763 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:24.655573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83152e-c9aa-41b1-83a7-8b6a64a9e576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:24.656225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83152e-c9aa-41b1-83a7-8b6a64a9e576 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.657068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83152e-c9aa-41b1-83a7-8b6a64a9e576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.657068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83152e-c9aa-41b1-83a7-8b6a64a9e576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.657567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83152e-c9aa-41b1-83a7-8b6a64a9e576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:24.659233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83152e-c9aa-41b1-83a7-8b6a64a9e576 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:24.677374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1146/2258] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:24.677374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83152e-c9aa-41b1-83a7-8b6a64a9e576 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:24.701582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83152e-c9aa-41b1-83a7-8b6a64a9e576 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:24.707439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a625b-e7d0-4775-a940-7e775fe298bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:24.708274 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a625b-e7d0-4775-a940-7e775fe298bd admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.708666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a625b-e7d0-4775-a940-7e775fe298bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.709105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a625b-e7d0-4775-a940-7e775fe298bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.710009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a625b-e7d0-4775-a940-7e775fe298bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:24.717159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a625b-e7d0-4775-a940-7e775fe298bd admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:24.726419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:24.740035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1147/2259] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:24.741451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a625b-e7d0-4775-a940-7e775fe298bd admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:24.756953 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a625b-e7d0-4775-a940-7e775fe298bd admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:24.774018 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808de9a9-ee8e-4013-8666-bca93ae6ef91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:24.774971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808de9a9-ee8e-4013-8666-bca93ae6ef91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.775258 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808de9a9-ee8e-4013-8666-bca93ae6ef91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.775539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808de9a9-ee8e-4013-8666-bca93ae6ef91 admin admin] PATH_INFO: `/v3/projects/940cf692c0554607a4d43de7c23a22a2/users/b28a7273be4947a18f690277ca61575e/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.779524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808de9a9-ee8e-4013-8666-bca93ae6ef91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=940cf692c0554607a4d43de7c23a22a2, user_id=b28a7273be4947a18f690277ca61575e, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:24.785223 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808de9a9-ee8e-4013-8666-bca93ae6ef91 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:24.810053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1148/2260] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:24 2022] PUT /identity/v3/projects/940cf692c0554607a4d43de7c23a22a2/users/b28a7273be4947a18f690277ca61575e/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:24.815736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808de9a9-ee8e-4013-8666-bca93ae6ef91 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:24.921563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e06158aa43645fb834b9c87b6d84409'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:24.958736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e06158aa43645fb834b9c87b6d84409'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:24.994184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808de9a9-ee8e-4013-8666-bca93ae6ef91 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:24.994771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e06158aa43645fb834b9c87b6d84409'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:25.005795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ef1b0-427f-4d72-b293-a719f2fdc94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:25.006509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ef1b0-427f-4d72-b293-a719f2fdc94b admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:25.006795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ef1b0-427f-4d72-b293-a719f2fdc94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:25.007076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ef1b0-427f-4d72-b293-a719f2fdc94b admin admin] PATH_INFO: `/v3/projects/4ac90963380143459e9a48e0d2b148a7/users/006a5fb3d01e42fc8042b0224ec37149/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:25.048957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ef1b0-427f-4d72-b293-a719f2fdc94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac90963380143459e9a48e0d2b148a7, user_id=006a5fb3d01e42fc8042b0224ec37149, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:25.058295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ef1b0-427f-4d72-b293-a719f2fdc94b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:25.117147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e06158aa43645fb834b9c87b6d84409'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:25.117877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8da55163-2e64-478c-af1f-78c1d5e4ac53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e06158aa43645fb834b9c87b6d84409'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:25.130529 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1113/2261] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:31:24 2022] DELETE /identity/v3/users/1e06158aa43645fb834b9c87b6d84409 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:25.132497 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1290e-d944-46bf-b831-07fc772667bb None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:25.132641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1290e-d944-46bf-b831-07fc772667bb None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:25.133030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1290e-d944-46bf-b831-07fc772667bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:25.158587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1149/2262] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:24 2022] PUT /identity/v3/projects/4ac90963380143459e9a48e0d2b148a7/users/006a5fb3d01e42fc8042b0224ec37149/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:25.165858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ef1b0-427f-4d72-b293-a719f2fdc94b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:25.171120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e90ef1b0-427f-4d72-b293-a719f2fdc94b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:25.177936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c8c1290e-d944-46bf-b831-07fc772667bb None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:25.184845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c8c1290e-d944-46bf-b831-07fc772667bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:25.335234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1114/2263] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:31:25.344171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1290e-d944-46bf-b831-07fc772667bb None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:25.347621 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c8c1290e-d944-46bf-b831-07fc772667bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:25.380266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ef1b0-427f-4d72-b293-a719f2fdc94b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:25.396185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2a7a4-e32e-44e7-aef7-cb1b9ee6c0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:25.398730 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2a7a4-e32e-44e7-aef7-cb1b9ee6c0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:25.413631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2a7a4-e32e-44e7-aef7-cb1b9ee6c0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:25.414085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2a7a4-e32e-44e7-aef7-cb1b9ee6c0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:25.415398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2a7a4-e32e-44e7-aef7-cb1b9ee6c0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:25.420285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2a7a4-e32e-44e7-aef7-cb1b9ee6c0a9 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:25.462788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1150/2264] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:25.467438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2a7a4-e32e-44e7-aef7-cb1b9ee6c0a9 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:25.473294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1290e-d944-46bf-b831-07fc772667bb None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:25.503601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2a7a4-e32e-44e7-aef7-cb1b9ee6c0a9 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:25.505610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed34de-10d2-4e3d-a96f-f1c702521e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:25.511790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed34de-10d2-4e3d-a96f-f1c702521e3c service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:25.512501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed34de-10d2-4e3d-a96f-f1c702521e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:25.513596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed34de-10d2-4e3d-a96f-f1c702521e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:25.522335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252052c3-e1a9-4b1b-9e92-fbd23725ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:25.523067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252052c3-e1a9-4b1b-9e92-fbd23725ef02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:25.523552 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252052c3-e1a9-4b1b-9e92-fbd23725ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:25.523842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252052c3-e1a9-4b1b-9e92-fbd23725ef02 admin admin] PATH_INFO: `/v3/projects/4ac90963380143459e9a48e0d2b148a7/users/006a5fb3d01e42fc8042b0224ec37149/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:25.530491 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252052c3-e1a9-4b1b-9e92-fbd23725ef02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac90963380143459e9a48e0d2b148a7, user_id=006a5fb3d01e42fc8042b0224ec37149, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:25.534579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252052c3-e1a9-4b1b-9e92-fbd23725ef02 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:25.547378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed34de-10d2-4e3d-a96f-f1c702521e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:25.554748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed34de-10d2-4e3d-a96f-f1c702521e3c service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:25.571786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1115/2265] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:25.594810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1151/2266] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:25 2022] PUT /identity/v3/projects/4ac90963380143459e9a48e0d2b148a7/users/006a5fb3d01e42fc8042b0224ec37149/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:27.218387 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed34de-10d2-4e3d-a96f-f1c702521e3c service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:27.220426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-eeed34de-10d2-4e3d-a96f-f1c702521e3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:27.234116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:27.242006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed34de-10d2-4e3d-a96f-f1c702521e3c service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:27.470194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:27.470883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:27.471175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:27.471436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] PATH_INFO: `/v3/projects/d0b889e950b644ac8a5119b28bbc2b61` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:27.473988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0b889e950b644ac8a5119b28bbc2b61)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:27.476315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:27.655758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0b889e950b644ac8a5119b28bbc2b61', 'request_id': 'req-410b0843-2b1f-4b6d-9863-9542f4efdebb'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:27.674401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0b889e950b644ac8a5119b28bbc2b61', 'request_id': 'req-410b0843-2b1f-4b6d-9863-9542f4efdebb'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:27.686673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0b889e950b644ac8a5119b28bbc2b61', 'request_id': 'req-410b0843-2b1f-4b6d-9863-9542f4efdebb'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:27.773611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] The token cache is being invalidate because project d0b889e950b644ac8a5119b28bbc2b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:31:27.774089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:27.776011 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:27.806413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1116/2267] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:31:27 2022] DELETE /identity/v3/projects/d0b889e950b644ac8a5119b28bbc2b61 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:28.441820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22cb11-c045-49e4-bd2d-25119af7eec1 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:28.442384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22cb11-c045-49e4-bd2d-25119af7eec1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:28.442723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22cb11-c045-49e4-bd2d-25119af7eec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:28.489529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-bc22cb11-c045-49e4-bd2d-25119af7eec1 None None] MFA Rules not processed for user `006a5fb3d01e42fc8042b0224ec37149`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:28.490856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-bc22cb11-c045-49e4-bd2d-25119af7eec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:28.572265 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:28.643344 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1152/2268] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:28.663022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:28.664570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:28.774046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b0843-2b1f-4b6d-9863-9542f4efdebb admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:28.780108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2894aa-09c7-4c33-96de-ace71c93e025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:28.780884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2894aa-09c7-4c33-96de-ace71c93e025 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:28.781614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2894aa-09c7-4c33-96de-ace71c93e025 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:28.782764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2894aa-09c7-4c33-96de-ace71c93e025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:28.785589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ff2894aa-09c7-4c33-96de-ace71c93e025 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:28.909217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2894aa-09c7-4c33-96de-ace71c93e025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:28.911666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2894aa-09c7-4c33-96de-ace71c93e025 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:28.924628 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1117/2269] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:31:28.965852 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:30.776677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22cb11-c045-49e4-bd2d-25119af7eec1 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:30.778217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-bc22cb11-c045-49e4-bd2d-25119af7eec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:30.790987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22cb11-c045-49e4-bd2d-25119af7eec1 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:30.795311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f2ce5-5eda-446b-8ac1-e9a18eaf6039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:30.795954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f2ce5-5eda-446b-8ac1-e9a18eaf6039 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:30.796432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f2ce5-5eda-446b-8ac1-e9a18eaf6039 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:30.796716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f2ce5-5eda-446b-8ac1-e9a18eaf6039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:30.809181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f2ce5-5eda-446b-8ac1-e9a18eaf6039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:30.810812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f2ce5-5eda-446b-8ac1-e9a18eaf6039 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:30.825978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1153/2270] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:31:32.854780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685624-0bdd-435f-9f5d-3d22a808956d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:32.855313 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685624-0bdd-435f-9f5d-3d22a808956d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:32.855562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685624-0bdd-435f-9f5d-3d22a808956d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:32.912941 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bf685624-0bdd-435f-9f5d-3d22a808956d None None] MFA Rules not processed for user `b28a7273be4947a18f690277ca61575e`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:32.914220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bf685624-0bdd-435f-9f5d-3d22a808956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:33.016526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1118/2271] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:33.092698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f2ce5-5eda-446b-8ac1-e9a18eaf6039 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:33.094339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e69f2ce5-5eda-446b-8ac1-e9a18eaf6039 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:33.183943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f2ce5-5eda-446b-8ac1-e9a18eaf6039 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:33.197866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a930efc-bde8-4a0f-8980-826af132d8fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:33.198652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a930efc-bde8-4a0f-8980-826af132d8fe service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:33.198949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a930efc-bde8-4a0f-8980-826af132d8fe service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:33.199264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a930efc-bde8-4a0f-8980-826af132d8fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:33.201334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8a930efc-bde8-4a0f-8980-826af132d8fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:33.275146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a930efc-bde8-4a0f-8980-826af132d8fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:33.276792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a930efc-bde8-4a0f-8980-826af132d8fe service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:33.285678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1154/2272] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:34.058929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f741c6c-997f-452c-80c7-4e71dc615433 None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:34.059492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f741c6c-997f-452c-80c7-4e71dc615433 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:34.059715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f741c6c-997f-452c-80c7-4e71dc615433 None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:34.061565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1119/2273] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:34.068257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466997e-b018-4c0e-9893-6f85b428f400 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:34.068555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466997e-b018-4c0e-9893-6f85b428f400 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:34.069015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466997e-b018-4c0e-9893-6f85b428f400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:34.095139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4466997e-b018-4c0e-9893-6f85b428f400 None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:34.096980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4466997e-b018-4c0e-9893-6f85b428f400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:34.119249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1155/2274] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:31:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:34.141437 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f741c6c-997f-452c-80c7-4e71dc615433 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:34.143197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5f741c6c-997f-452c-80c7-4e71dc615433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:34.209451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f741c6c-997f-452c-80c7-4e71dc615433 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:34.215742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df43b5-5916-4c0c-9b95-9aa84bd569b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4fe04b11ec412e8f4eb67c6e54b8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:34.216482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df43b5-5916-4c0c-9b95-9aa84bd569b8 service glance] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:34.219052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df43b5-5916-4c0c-9b95-9aa84bd569b8 service glance] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:34.219459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df43b5-5916-4c0c-9b95-9aa84bd569b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:34.231483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df43b5-5916-4c0c-9b95-9aa84bd569b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:34.232938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df43b5-5916-4c0c-9b95-9aa84bd569b8 service glance] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:34.245282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1120/2275] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:34.634793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46305ce-f0a1-4dff-9121-e3f4632aea8c None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:34.635087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46305ce-f0a1-4dff-9121-e3f4632aea8c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:34.635371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46305ce-f0a1-4dff-9121-e3f4632aea8c None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:34.636779 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1156/2276] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:34.642232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852559a1-926d-4b2d-a8ea-8ebf2b92dc6b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:34.642522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852559a1-926d-4b2d-a8ea-8ebf2b92dc6b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:34.642801 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852559a1-926d-4b2d-a8ea-8ebf2b92dc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:34.668668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-852559a1-926d-4b2d-a8ea-8ebf2b92dc6b None None] MFA Rules not processed for user `05574eace38f4b6d945c7b1927fdadd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:34.670155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-852559a1-926d-4b2d-a8ea-8ebf2b92dc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:34.728397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1121/2277] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:31:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:34.741241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46305ce-f0a1-4dff-9121-e3f4632aea8c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:34.743910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b46305ce-f0a1-4dff-9121-e3f4632aea8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:34.759745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46305ce-f0a1-4dff-9121-e3f4632aea8c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:34.765591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b744958-7f5b-4756-a4e2-6c7819bbb43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:34.766281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b744958-7f5b-4756-a4e2-6c7819bbb43e service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:34.766571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b744958-7f5b-4756-a4e2-6c7819bbb43e service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:34.766853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b744958-7f5b-4756-a4e2-6c7819bbb43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:34.775813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b744958-7f5b-4756-a4e2-6c7819bbb43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:34.777283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b744958-7f5b-4756-a4e2-6c7819bbb43e service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:34.786000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1157/2278] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:34.879216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528ca6c-009e-4bf0-ad07-08ba9008757c None None] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:34.879216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528ca6c-009e-4bf0-ad07-08ba9008757c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:34.879216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528ca6c-009e-4bf0-ad07-08ba9008757c None None] PATH_INFO: `/` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:34.880713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1122/2279] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:34.891999 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571cc6c-b1f2-464a-bf8a-425d372ccec7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:34.892338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571cc6c-b1f2-464a-bf8a-425d372ccec7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:34.892569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571cc6c-b1f2-464a-bf8a-425d372ccec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:34.928397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-b571cc6c-b1f2-464a-bf8a-425d372ccec7 None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:34.930055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b571cc6c-b1f2-464a-bf8a-425d372ccec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:34.944293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1158/2280] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:31:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:31:34.956439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528ca6c-009e-4bf0-ad07-08ba9008757c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:34.958420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a528ca6c-009e-4bf0-ad07-08ba9008757c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:34.975480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528ca6c-009e-4bf0-ad07-08ba9008757c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:34.980415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c11e8-1491-4e51-98b0-5ecf7c335c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:34.982320 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c11e8-1491-4e51-98b0-5ecf7c335c72 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:34.983077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c11e8-1491-4e51-98b0-5ecf7c335c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:34.983438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c11e8-1491-4e51-98b0-5ecf7c335c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:34.997151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c11e8-1491-4e51-98b0-5ecf7c335c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:34.999015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c11e8-1491-4e51-98b0-5ecf7c335c72 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:35.016047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1123/2281] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:44.920519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=84 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:45.006964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:48.062786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571cc6c-b1f2-464a-bf8a-425d372ccec7 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:48.064269 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b571cc6c-b1f2-464a-bf8a-425d372ccec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:48.137058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571cc6c-b1f2-464a-bf8a-425d372ccec7 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:48.141910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:48.142572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:48.142889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:48.143152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] PATH_INFO: `/v3/users/061889c63b65453f8c0d409acfab420e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:48.144342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=061889c63b65453f8c0d409acfab420e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:48.146320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:48.326782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '061889c63b65453f8c0d409acfab420e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:48.342027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '061889c63b65453f8c0d409acfab420e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:48.342938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '061889c63b65453f8c0d409acfab420e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:48.358738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '061889c63b65453f8c0d409acfab420e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:48.461011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '061889c63b65453f8c0d409acfab420e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:48.468381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1159/2282] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:31:48 2022] DELETE /identity/v3/users/061889c63b65453f8c0d409acfab420e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:48.721822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c11e8-1491-4e51-98b0-5ecf7c335c72 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:48.734252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c11e8-1491-4e51-98b0-5ecf7c335c72 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:48.815690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:48.816270 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:48.816553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:48.816831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] PATH_INFO: `/v3/projects/6f42ddc1f920465f88baa24820a3976f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:48.818005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f42ddc1f920465f88baa24820a3976f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:48.819923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:48.954385 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f42ddc1f920465f88baa24820a3976f', 'request_id': 'req-81ea52fb-c816-4a3d-9b97-f9a55f242203'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:48.960372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f42ddc1f920465f88baa24820a3976f', 'request_id': 'req-81ea52fb-c816-4a3d-9b97-f9a55f242203'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:48.968527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f42ddc1f920465f88baa24820a3976f', 'request_id': 'req-81ea52fb-c816-4a3d-9b97-f9a55f242203'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:49.038052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] The token cache is being invalidate because project 6f42ddc1f920465f88baa24820a3976f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:31:49.038349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:49.039005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:49.041324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1124/2283] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:31:48 2022] DELETE /identity/v3/projects/6f42ddc1f920465f88baa24820a3976f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:49.045545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:49.052768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:49.060906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259052fb-70df-4de4-be31-838ad2d5b5ea admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:49.122935 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:49.123613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:49.123863 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:49.124145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] PATH_INFO: `/v3/users/006a5fb3d01e42fc8042b0224ec37149` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:49.125334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=006a5fb3d01e42fc8042b0224ec37149)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:49.127370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:49.149070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:49.380184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006a5fb3d01e42fc8042b0224ec37149'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:49.395184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006a5fb3d01e42fc8042b0224ec37149'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:49.396073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006a5fb3d01e42fc8042b0224ec37149'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:49.406219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006a5fb3d01e42fc8042b0224ec37149'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:49.478903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7e2295ac-a376-49a8-a1cd-b4efded44087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006a5fb3d01e42fc8042b0224ec37149'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:49.485945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1160/2284] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:31:49 2022] DELETE /identity/v3/users/006a5fb3d01e42fc8042b0224ec37149 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:31:50.014963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:50.014963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:50.029585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea52fb-c816-4a3d-9b97-f9a55f242203 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:50.089359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:50.089940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:50.090225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:50.090506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] PATH_INFO: `/v3/projects/4ac90963380143459e9a48e0d2b148a7` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:50.091678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ac90963380143459e9a48e0d2b148a7)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:50.093873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:50.207147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:50.222238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ac90963380143459e9a48e0d2b148a7', 'request_id': 'req-77f44fab-fb63-401a-afe7-b894ddaee0df'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:50.227915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ac90963380143459e9a48e0d2b148a7', 'request_id': 'req-77f44fab-fb63-401a-afe7-b894ddaee0df'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:50.234472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ac90963380143459e9a48e0d2b148a7', 'request_id': 'req-77f44fab-fb63-401a-afe7-b894ddaee0df'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:50.309859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] The token cache is being invalidate because project 4ac90963380143459e9a48e0d2b148a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:31:50.310147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:50.310782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-77f44fab-fb63-401a-afe7-b894ddaee0df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:31:50.312346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1125/2285] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:31:50 2022] DELETE /identity/v3/projects/4ac90963380143459e9a48e0d2b148a7 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:50.317581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472af0f-16e8-4b87-b140-1c6a7110ba81 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:50.318915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472af0f-16e8-4b87-b140-1c6a7110ba81 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:50.319162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472af0f-16e8-4b87-b140-1c6a7110ba81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:50.340445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-7472af0f-16e8-4b87-b140-1c6a7110ba81 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:50.341996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7472af0f-16e8-4b87-b140-1c6a7110ba81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:50.423306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1161/2286] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:50.424651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cc619-48e8-4a02-9f21-ad38f791ac90 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:50.424984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cc619-48e8-4a02-9f21-ad38f791ac90 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:50.425274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cc619-48e8-4a02-9f21-ad38f791ac90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:50.446157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e01cc619-48e8-4a02-9f21-ad38f791ac90 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:50.448083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e01cc619-48e8-4a02-9f21-ad38f791ac90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:50.456598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1126/2287] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:50.460997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472af0f-16e8-4b87-b140-1c6a7110ba81 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:50.471752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472af0f-16e8-4b87-b140-1c6a7110ba81 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:50.476573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32257da-ed1a-4bc7-bf74-5f0a424a1027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:50.477237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32257da-ed1a-4bc7-bf74-5f0a424a1027 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:50.477557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32257da-ed1a-4bc7-bf74-5f0a424a1027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:50.477985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32257da-ed1a-4bc7-bf74-5f0a424a1027 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:50.478828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32257da-ed1a-4bc7-bf74-5f0a424a1027 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:31:50.479142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32257da-ed1a-4bc7-bf74-5f0a424a1027 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:50.480783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32257da-ed1a-4bc7-bf74-5f0a424a1027 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:50.498422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1162/2288] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:31:50.501683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cc619-48e8-4a02-9f21-ad38f791ac90 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:50.510399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cc619-48e8-4a02-9f21-ad38f791ac90 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:50.515408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c8e3c-3979-4bbb-88fb-0b000288508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:50.516031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8e3c-3979-4bbb-88fb-0b000288508d admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:50.516331 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8e3c-3979-4bbb-88fb-0b000288508d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:50.516606 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8e3c-3979-4bbb-88fb-0b000288508d admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:50.517285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c8e3c-3979-4bbb-88fb-0b000288508d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:50.519303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c8e3c-3979-4bbb-88fb-0b000288508d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:50.637501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1127/2289] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:31:50 2022] POST /identity/v3/projects => generated 391 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:31:50.642534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32257da-ed1a-4bc7-bf74-5f0a424a1027 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:50.652694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32257da-ed1a-4bc7-bf74-5f0a424a1027 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:50.708773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76836e96-e510-4696-a534-f6c5abc75761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:50.709419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76836e96-e510-4696-a534-f6c5abc75761 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:50.709705 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76836e96-e510-4696-a534-f6c5abc75761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:50.709982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76836e96-e510-4696-a534-f6c5abc75761 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:50.710665 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76836e96-e510-4696-a534-f6c5abc75761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:50.713654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76836e96-e510-4696-a534-f6c5abc75761 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:50.815906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1163/2290] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:31:50 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:31:50.819805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c8e3c-3979-4bbb-88fb-0b000288508d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:50.830862 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c8e3c-3979-4bbb-88fb-0b000288508d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:50.835984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e0d4d-3779-4969-a12e-02c0f79bcc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:50.836628 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e0d4d-3779-4969-a12e-02c0f79bcc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:50.839376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e0d4d-3779-4969-a12e-02c0f79bcc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:50.839376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e0d4d-3779-4969-a12e-02c0f79bcc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:50.839376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e0d4d-3779-4969-a12e-02c0f79bcc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:50.839836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e0d4d-3779-4969-a12e-02c0f79bcc3d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:50.850939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1128/2291] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:50.855601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76836e96-e510-4696-a534-f6c5abc75761 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:50.865893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76836e96-e510-4696-a534-f6c5abc75761 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:50.871248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b1bb9-e777-49f5-94f6-ba59fd5379c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:50.871860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b1bb9-e777-49f5-94f6-ba59fd5379c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:50.872152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b1bb9-e777-49f5-94f6-ba59fd5379c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:50.872431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b1bb9-e777-49f5-94f6-ba59fd5379c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:50.873178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b1bb9-e777-49f5-94f6-ba59fd5379c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:50.874817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b1bb9-e777-49f5-94f6-ba59fd5379c8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:50.886018 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1164/2292] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:50.890337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e0d4d-3779-4969-a12e-02c0f79bcc3d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:50.901339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e0d4d-3779-4969-a12e-02c0f79bcc3d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:50.906160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299fbe6-21b4-4b24-bca2-30ca8eb46433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:50.913722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299fbe6-21b4-4b24-bca2-30ca8eb46433 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:50.913722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299fbe6-21b4-4b24-bca2-30ca8eb46433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:50.913722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299fbe6-21b4-4b24-bca2-30ca8eb46433 admin admin] PATH_INFO: `/v3/projects/c3f06b92b07f43c9a4a04af3044899de/users/844e64d7095e462ba8046dc707cf695b/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:50.923159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299fbe6-21b4-4b24-bca2-30ca8eb46433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f06b92b07f43c9a4a04af3044899de, user_id=844e64d7095e462ba8046dc707cf695b, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:50.925897 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299fbe6-21b4-4b24-bca2-30ca8eb46433 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:50.978962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1129/2293] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:31:50 2022] PUT /identity/v3/projects/c3f06b92b07f43c9a4a04af3044899de/users/844e64d7095e462ba8046dc707cf695b/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:31:50.991655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded81a83-618f-4b8f-8d74-152f1fce2a94 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:50.992024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded81a83-618f-4b8f-8d74-152f1fce2a94 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:50.992410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded81a83-618f-4b8f-8d74-152f1fce2a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:51.001643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:51.028835 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-ded81a83-618f-4b8f-8d74-152f1fce2a94 None None] MFA Rules not processed for user `844e64d7095e462ba8046dc707cf695b`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:51.030256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ded81a83-618f-4b8f-8d74-152f1fce2a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:51.111500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1165/2294] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:51.119659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299fbe6-21b4-4b24-bca2-30ca8eb46433 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:51.122045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f299fbe6-21b4-4b24-bca2-30ca8eb46433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:51.191163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299fbe6-21b4-4b24-bca2-30ca8eb46433 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:51.195719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f20fb-daea-4c75-9c89-f576da01652e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:51.195719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f20fb-daea-4c75-9c89-f576da01652e service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:51.195980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f20fb-daea-4c75-9c89-f576da01652e service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:51.196206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f20fb-daea-4c75-9c89-f576da01652e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:51.205837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f20fb-daea-4c75-9c89-f576da01652e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:51.207381 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f20fb-daea-4c75-9c89-f576da01652e service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:51.217087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1130/2295] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:58.098417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16c797-ad88-4950-b989-9548bbc4bb19 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:58.098882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16c797-ad88-4950-b989-9548bbc4bb19 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:58.098971 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16c797-ad88-4950-b989-9548bbc4bb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:58.119065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-7f16c797-ad88-4950-b989-9548bbc4bb19 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:58.120509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7f16c797-ad88-4950-b989-9548bbc4bb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:58.186343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1166/2296] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:31:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:58.199354 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f20fb-daea-4c75-9c89-f576da01652e service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:58.200835 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cb4f20fb-daea-4c75-9c89-f576da01652e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:58.280092 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:58.286793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f20fb-daea-4c75-9c89-f576da01652e service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:58.292289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16998aa5-da33-47ee-a9d5-d7fdb8093a74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:58.292923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16998aa5-da33-47ee-a9d5-d7fdb8093a74 service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:58.293213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16998aa5-da33-47ee-a9d5-d7fdb8093a74 service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:58.293535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16998aa5-da33-47ee-a9d5-d7fdb8093a74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:58.302482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16998aa5-da33-47ee-a9d5-d7fdb8093a74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:58.304147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16998aa5-da33-47ee-a9d5-d7fdb8093a74 service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:58.313203 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1131/2297] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:31:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:32:08.176756 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=55 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:32:08.366311 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:32:13.845638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a2207-8e02-4558-b5f2-8b77fdf166ce None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:32:13.846260 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a2207-8e02-4558-b5f2-8b77fdf166ce None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:32:13.846368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a2207-8e02-4558-b5f2-8b77fdf166ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:32:13.867051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2f9a2207-8e02-4558-b5f2-8b77fdf166ce None None] MFA Rules not processed for user `b1868c7d0ec8443b9ccb698ee02928b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:32:13.868408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2f9a2207-8e02-4558-b5f2-8b77fdf166ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:32:13.927614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1167/2298] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:32:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:32:15.408660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16998aa5-da33-47ee-a9d5-d7fdb8093a74 service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:32:15.480920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16998aa5-da33-47ee-a9d5-d7fdb8093a74 service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:32:15.485628 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:32:15.486291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:32:15.486581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:32:15.486864 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] PATH_INFO: `/v3/users/844e64d7095e462ba8046dc707cf695b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:32:15.488091 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=844e64d7095e462ba8046dc707cf695b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:32:15.490182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:32:15.674399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '844e64d7095e462ba8046dc707cf695b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:32:15.692822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '844e64d7095e462ba8046dc707cf695b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:32:15.703067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '844e64d7095e462ba8046dc707cf695b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:32:15.806662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '844e64d7095e462ba8046dc707cf695b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:32:15.808068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '844e64d7095e462ba8046dc707cf695b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:32:15.815505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1132/2299] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:32:15 2022] DELETE /identity/v3/users/844e64d7095e462ba8046dc707cf695b => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:32:15.819929 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f7e8-de3a-4abe-8aac-d1b65f577af2 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:32:15.820480 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f7e8-de3a-4abe-8aac-d1b65f577af2 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:32:15.820570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f7e8-de3a-4abe-8aac-d1b65f577af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:32:15.845753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4ac5f7e8-de3a-4abe-8aac-d1b65f577af2 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:32:15.847088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4ac5f7e8-de3a-4abe-8aac-d1b65f577af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:32:15.921299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1168/2300] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:32:15.928545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:32:15.930100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:32:16.056365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d74cd0-a225-449f-8067-5248e5154ffe admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:32:16.062112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf0144-7b3d-462f-87c6-18db0268eb78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:32:16.062770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf0144-7b3d-462f-87c6-18db0268eb78 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:32:16.063058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf0144-7b3d-462f-87c6-18db0268eb78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:32:16.063340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf0144-7b3d-462f-87c6-18db0268eb78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:32:16.076164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf0144-7b3d-462f-87c6-18db0268eb78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:32:16.077586 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf0144-7b3d-462f-87c6-18db0268eb78 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:32:16.092400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1133/2301] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:32:16.761025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5f7e8-de3a-4abe-8aac-d1b65f577af2 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:32:16.762613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4ac5f7e8-de3a-4abe-8aac-d1b65f577af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:32:16.784463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5f7e8-de3a-4abe-8aac-d1b65f577af2 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:32:16.796179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:32:16.797094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:32:16.797520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:32:16.797920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] PATH_INFO: `/v3/projects/c3f06b92b07f43c9a4a04af3044899de` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:32:16.799444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f06b92b07f43c9a4a04af3044899de)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:32:16.801817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:32:16.950793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f06b92b07f43c9a4a04af3044899de', 'request_id': 'req-74de779f-fa77-4683-bec4-92924f5a9008'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:32:16.965198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f06b92b07f43c9a4a04af3044899de', 'request_id': 'req-74de779f-fa77-4683-bec4-92924f5a9008'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:32:17.033488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f06b92b07f43c9a4a04af3044899de', 'request_id': 'req-74de779f-fa77-4683-bec4-92924f5a9008'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:32:17.053192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] The token cache is being invalidate because project c3f06b92b07f43c9a4a04af3044899de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:32:17.053553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:32:17.054231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:32:17.056534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1169/2302] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:32:16 2022] DELETE /identity/v3/projects/c3f06b92b07f43c9a4a04af3044899de => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:32:21.500947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741445f9-b261-439f-a561-078ee9dbbf37 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:32:21.501501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741445f9-b261-439f-a561-078ee9dbbf37 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:32:21.501617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741445f9-b261-439f-a561-078ee9dbbf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:32:21.522716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-741445f9-b261-439f-a561-078ee9dbbf37 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:32:21.523947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-741445f9-b261-439f-a561-078ee9dbbf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:32:21.544546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:32:21.587581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1134/2303] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 28 06:32:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:32:21.602349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:32:21.603859 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:32:21.639089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:32:21.676511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de779f-fa77-4683-bec4-92924f5a9008 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:32:21.681846 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f437296-5f83-4200-b335-986117dafb36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:32:21.682492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f437296-5f83-4200-b335-986117dafb36 service swift] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:32:21.682782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f437296-5f83-4200-b335-986117dafb36 service swift] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:32:21.683065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f437296-5f83-4200-b335-986117dafb36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:32:21.691981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f437296-5f83-4200-b335-986117dafb36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:32:21.693721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f437296-5f83-4200-b335-986117dafb36 service swift] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:32:21.702989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1170/2304] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:32:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:32:32.046462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=15 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:32:32.243283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=19 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:32:41.637871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48cdd-0d57-4f43-88a1-600e9657b712 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:32:41.638472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48cdd-0d57-4f43-88a1-600e9657b712 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:32:41.638472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48cdd-0d57-4f43-88a1-600e9657b712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:32:41.659741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-27a48cdd-0d57-4f43-88a1-600e9657b712 None None] MFA Rules not processed for user `86a5553819d949ffbc64b7a388942baf`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:32:41.663836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-27a48cdd-0d57-4f43-88a1-600e9657b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:32:41.721728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1135/2305] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:32:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:32:43.364157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795c249-5253-4d59-90e9-7dab5f8fa73a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:32:43.364748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795c249-5253-4d59-90e9-7dab5f8fa73a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:32:43.364748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795c249-5253-4d59-90e9-7dab5f8fa73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:32:43.384516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f795c249-5253-4d59-90e9-7dab5f8fa73a None None] MFA Rules not processed for user `61ad8186d91848bc8be42e3be51e9a75`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:32:43.385753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f795c249-5253-4d59-90e9-7dab5f8fa73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:32:43.445497 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1171/2306] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:32:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:32:50.568149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a48cdd-0d57-4f43-88a1-600e9657b712 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:32:50.569682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-27a48cdd-0d57-4f43-88a1-600e9657b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:32:50.639518 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a48cdd-0d57-4f43-88a1-600e9657b712 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:32:50.644899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340c92-c838-453a-9441-d46ef6b947b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:32:50.645764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340c92-c838-453a-9441-d46ef6b947b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:32:50.646057 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340c92-c838-453a-9441-d46ef6b947b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:32:50.646481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340c92-c838-453a-9441-d46ef6b947b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:32:50.648691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4e340c92-c838-453a-9441-d46ef6b947b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:32:50.751375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e340c92-c838-453a-9441-d46ef6b947b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:32:50.752914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e340c92-c838-453a-9441-d46ef6b947b6 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:32:50.764127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1136/2307] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:32:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:32:53.437701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=25 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:00.758918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=75 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:01.301017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f77bad-5059-459e-b0fb-63030101bc4c None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:01.301571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f77bad-5059-459e-b0fb-63030101bc4c None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:01.301669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f77bad-5059-459e-b0fb-63030101bc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:01.321613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-94f77bad-5059-459e-b0fb-63030101bc4c None None] MFA Rules not processed for user `97547d56a8684009a794a42ad1c5f489`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:33:01.322835 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-94f77bad-5059-459e-b0fb-63030101bc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:01.380620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1172/2308] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:33:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:33:08.570144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340c92-c838-453a-9441-d46ef6b947b6 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:08.571660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4e340c92-c838-453a-9441-d46ef6b947b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:08.583330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340c92-c838-453a-9441-d46ef6b947b6 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:08.587258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ccbc3-4294-4e7b-8058-93be600eb2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:08.587865 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ccbc3-4294-4e7b-8058-93be600eb2eb service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:08.588147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ccbc3-4294-4e7b-8058-93be600eb2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:08.588424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ccbc3-4294-4e7b-8058-93be600eb2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:08.590173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f77ccbc3-4294-4e7b-8058-93be600eb2eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:08.670736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ccbc3-4294-4e7b-8058-93be600eb2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:08.672117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ccbc3-4294-4e7b-8058-93be600eb2eb service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:08.683406 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1137/2309] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:33:09.354896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f77bad-5059-459e-b0fb-63030101bc4c None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:09.371121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f77bad-5059-459e-b0fb-63030101bc4c None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:09.375922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf96812-eafb-4503-8371-b8969ed7fbd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:09.376563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf96812-eafb-4503-8371-b8969ed7fbd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:09.376851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf96812-eafb-4503-8371-b8969ed7fbd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:09.377132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf96812-eafb-4503-8371-b8969ed7fbd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:09.379011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-baf96812-eafb-4503-8371-b8969ed7fbd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:09.393579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf96812-eafb-4503-8371-b8969ed7fbd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:09.395044 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf96812-eafb-4503-8371-b8969ed7fbd1 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:09.407462 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1173/2310] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:33:17.285271 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a607a63-4496-4f11-8969-16964db7de93 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:17.285810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a607a63-4496-4f11-8969-16964db7de93 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:17.285931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a607a63-4496-4f11-8969-16964db7de93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:17.306616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0a607a63-4496-4f11-8969-16964db7de93 None None] MFA Rules not processed for user `6d4528589ffa478689e905d47e4c37ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:33:17.307843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0a607a63-4496-4f11-8969-16964db7de93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:17.369049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1138/2311] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:33:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:33:19.401712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=31 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:27.359165 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=59 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:28.209097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf96812-eafb-4503-8371-b8969ed7fbd1 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:28.210518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-baf96812-eafb-4503-8371-b8969ed7fbd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:28.270986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf96812-eafb-4503-8371-b8969ed7fbd1 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:28.274967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1015e10-2da0-4e70-a8e4-5e507e3a65fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:28.275612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1015e10-2da0-4e70-a8e4-5e507e3a65fd service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:28.275897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1015e10-2da0-4e70-a8e4-5e507e3a65fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:28.276179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1015e10-2da0-4e70-a8e4-5e507e3a65fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:28.277968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e1015e10-2da0-4e70-a8e4-5e507e3a65fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:28.290870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1015e10-2da0-4e70-a8e4-5e507e3a65fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:28.297860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1015e10-2da0-4e70-a8e4-5e507e3a65fd service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:28.301934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1174/2312] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:33:28.637838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a607a63-4496-4f11-8969-16964db7de93 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:28.639341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0a607a63-4496-4f11-8969-16964db7de93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:28.655374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a607a63-4496-4f11-8969-16964db7de93 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:28.661206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e84a1-15e8-4842-9462-7a77ebee61e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:28.661822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e84a1-15e8-4842-9462-7a77ebee61e4 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:28.662075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e84a1-15e8-4842-9462-7a77ebee61e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:28.662355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e84a1-15e8-4842-9462-7a77ebee61e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:28.664212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9c2e84a1-15e8-4842-9462-7a77ebee61e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:28.676093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e84a1-15e8-4842-9462-7a77ebee61e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:28.677883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e84a1-15e8-4842-9462-7a77ebee61e4 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:28.688409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1139/2313] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:33:36.849736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1015e10-2da0-4e70-a8e4-5e507e3a65fd service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:36.858547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1015e10-2da0-4e70-a8e4-5e507e3a65fd service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:36.863163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197382f-b89d-4705-84db-a77ff64ff6b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:36.863791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197382f-b89d-4705-84db-a77ff64ff6b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:36.864081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197382f-b89d-4705-84db-a77ff64ff6b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:36.864367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197382f-b89d-4705-84db-a77ff64ff6b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:36.866339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f197382f-b89d-4705-84db-a77ff64ff6b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:36.930869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197382f-b89d-4705-84db-a77ff64ff6b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:36.932331 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197382f-b89d-4705-84db-a77ff64ff6b1 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:36.942285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1175/2314] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:33:38.683314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:43.228503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e84a1-15e8-4842-9462-7a77ebee61e4 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:43.238022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e84a1-15e8-4842-9462-7a77ebee61e4 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:43.241837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0bae1-84f9-429b-9f18-ede213eb3213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:43.242348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0bae1-84f9-429b-9f18-ede213eb3213 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:43.242661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0bae1-84f9-429b-9f18-ede213eb3213 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:43.242948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0bae1-84f9-429b-9f18-ede213eb3213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:43.244722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-39a0bae1-84f9-429b-9f18-ede213eb3213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:43.255543 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0bae1-84f9-429b-9f18-ede213eb3213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:43.256991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0bae1-84f9-429b-9f18-ede213eb3213 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:43.266871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1140/2315] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:33:46.455785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197382f-b89d-4705-84db-a77ff64ff6b1 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:46.457662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f197382f-b89d-4705-84db-a77ff64ff6b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:46.498221 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197382f-b89d-4705-84db-a77ff64ff6b1 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:46.508348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e862e39-9503-4954-babf-f141be19269e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:46.509528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e862e39-9503-4954-babf-f141be19269e service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:46.510574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e862e39-9503-4954-babf-f141be19269e service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:46.511101 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e862e39-9503-4954-babf-f141be19269e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:46.514945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8e862e39-9503-4954-babf-f141be19269e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:46.534603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e862e39-9503-4954-babf-f141be19269e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:46.537146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e862e39-9503-4954-babf-f141be19269e service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:46.573205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1176/2316] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:33:52.073665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b08875-2d69-4c45-a976-a2b485d07f12 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:52.074330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b08875-2d69-4c45-a976-a2b485d07f12 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:52.074330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b08875-2d69-4c45-a976-a2b485d07f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:52.092713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e862e39-9503-4954-babf-f141be19269e service nova] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:52.095008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-62b08875-2d69-4c45-a976-a2b485d07f12 None None] MFA Rules not processed for user `0d268e37c8b64bcc9261a80bb0461edc`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:33:52.096222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-62b08875-2d69-4c45-a976-a2b485d07f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:52.098603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=61 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:52.103744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1141/2317] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:33:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:33:52.104700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e862e39-9503-4954-babf-f141be19269e service nova] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:52.110753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82e04c-5bab-4424-81c6-77a2a812a76c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:52.111392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82e04c-5bab-4424-81c6-77a2a812a76c service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:52.111679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82e04c-5bab-4424-81c6-77a2a812a76c service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:52.111963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82e04c-5bab-4424-81c6-77a2a812a76c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:52.113988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-bb82e04c-5bab-4424-81c6-77a2a812a76c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:52.126777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82e04c-5bab-4424-81c6-77a2a812a76c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:52.128384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82e04c-5bab-4424-81c6-77a2a812a76c service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:52.139128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1177/2318] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:33:52.425131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b08875-2d69-4c45-a976-a2b485d07f12 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:52.435095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b08875-2d69-4c45-a976-a2b485d07f12 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:52.440224 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efb61f-df2d-4c60-a725-f7cb4058917a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:52.440770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efb61f-df2d-4c60-a725-f7cb4058917a service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:52.441056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efb61f-df2d-4c60-a725-f7cb4058917a service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:52.441372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efb61f-df2d-4c60-a725-f7cb4058917a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:52.443151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-83efb61f-df2d-4c60-a725-f7cb4058917a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:52.453391 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efb61f-df2d-4c60-a725-f7cb4058917a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:52.454857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efb61f-df2d-4c60-a725-f7cb4058917a service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:52.463348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1142/2319] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:33:52.853117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82e04c-5bab-4424-81c6-77a2a812a76c service nova] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:52.854639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-bb82e04c-5bab-4424-81c6-77a2a812a76c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:52.942047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82e04c-5bab-4424-81c6-77a2a812a76c service nova] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:52.947434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88809c6-5521-4264-96c5-5c95bb42bacf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:52.948005 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88809c6-5521-4264-96c5-5c95bb42bacf service placement] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:52.954655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88809c6-5521-4264-96c5-5c95bb42bacf service placement] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:52.954655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88809c6-5521-4264-96c5-5c95bb42bacf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:52.954655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a88809c6-5521-4264-96c5-5c95bb42bacf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:52.962755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88809c6-5521-4264-96c5-5c95bb42bacf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:52.964366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88809c6-5521-4264-96c5-5c95bb42bacf service placement] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:52.975425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1178/2320] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:02.459426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=15 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:02.970905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=37 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:07.538364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efb61f-df2d-4c60-a725-f7cb4058917a service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:07.546989 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efb61f-df2d-4c60-a725-f7cb4058917a service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:07.556394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473f7cb-ec1b-4de8-9077-52da7c3cc3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:07.556394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473f7cb-ec1b-4de8-9077-52da7c3cc3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:07.556394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473f7cb-ec1b-4de8-9077-52da7c3cc3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:07.556394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473f7cb-ec1b-4de8-9077-52da7c3cc3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:07.556879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8473f7cb-ec1b-4de8-9077-52da7c3cc3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:07.623085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473f7cb-ec1b-4de8-9077-52da7c3cc3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:07.624592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473f7cb-ec1b-4de8-9077-52da7c3cc3b9 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:07.635411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1143/2321] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:11.703128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88809c6-5521-4264-96c5-5c95bb42bacf service placement] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:11.729411 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88809c6-5521-4264-96c5-5c95bb42bacf service placement] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:11.739673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b160f22-9d32-4991-904a-06068d3c0f3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:11.740582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b160f22-9d32-4991-904a-06068d3c0f3d service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:11.741078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b160f22-9d32-4991-904a-06068d3c0f3d service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:11.741579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b160f22-9d32-4991-904a-06068d3c0f3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:11.743666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-5b160f22-9d32-4991-904a-06068d3c0f3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:11.759534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b160f22-9d32-4991-904a-06068d3c0f3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:11.761355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b160f22-9d32-4991-904a-06068d3c0f3d service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:11.784540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1179/2322] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:11.917597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473f7cb-ec1b-4de8-9077-52da7c3cc3b9 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:11.919757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8473f7cb-ec1b-4de8-9077-52da7c3cc3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:11.935245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473f7cb-ec1b-4de8-9077-52da7c3cc3b9 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:11.940416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:11.941121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:11.941514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:11.942002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] PATH_INFO: `/v3/users/86a5553819d949ffbc64b7a388942baf` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:11.943630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a5553819d949ffbc64b7a388942baf)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:11.945811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:12.080784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a5553819d949ffbc64b7a388942baf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:12.102507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a5553819d949ffbc64b7a388942baf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:12.112408 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a5553819d949ffbc64b7a388942baf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:12.207431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a5553819d949ffbc64b7a388942baf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:12.208171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5d057f54-2699-447f-a53d-84562296fd2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a5553819d949ffbc64b7a388942baf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:12.215992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1144/2323] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:34:11 2022] DELETE /identity/v3/users/86a5553819d949ffbc64b7a388942baf => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:12.641537 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b160f22-9d32-4991-904a-06068d3c0f3d service nova] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:12.657242 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b160f22-9d32-4991-904a-06068d3c0f3d service nova] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:12.748968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:12.750173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:12.750471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:12.750755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] PATH_INFO: `/v3/projects/cd88554db98441c99e0b09b446c24b47` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:12.752010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd88554db98441c99e0b09b446c24b47)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:12.754008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:12.863976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd88554db98441c99e0b09b446c24b47', 'request_id': 'req-2040b40c-ac0a-447b-8202-d7011d9378f1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:12.869654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd88554db98441c99e0b09b446c24b47', 'request_id': 'req-2040b40c-ac0a-447b-8202-d7011d9378f1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:12.985321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd88554db98441c99e0b09b446c24b47', 'request_id': 'req-2040b40c-ac0a-447b-8202-d7011d9378f1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:13.000803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] The token cache is being invalidate because project cd88554db98441c99e0b09b446c24b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:34:13.001103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:13.001801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2040b40c-ac0a-447b-8202-d7011d9378f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:13.003362 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1180/2324] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:34:12 2022] DELETE /identity/v3/projects/cd88554db98441c99e0b09b446c24b47 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:13.008538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0475fc-bfaa-4085-ab67-ca3cb6ddc78d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:13.010441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0475fc-bfaa-4085-ab67-ca3cb6ddc78d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:13.010441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0475fc-bfaa-4085-ab67-ca3cb6ddc78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:13.030180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2a0475fc-bfaa-4085-ab67-ca3cb6ddc78d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:13.032727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2a0475fc-bfaa-4085-ab67-ca3cb6ddc78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:13.097682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1145/2325] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:13.111229 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c2c5c-4d94-477c-84df-7a4fdb9d69e3 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:13.111520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c2c5c-4d94-477c-84df-7a4fdb9d69e3 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:13.111801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c2c5c-4d94-477c-84df-7a4fdb9d69e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:13.131974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-440c2c5c-4d94-477c-84df-7a4fdb9d69e3 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:13.133259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-440c2c5c-4d94-477c-84df-7a4fdb9d69e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:13.142638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1181/2326] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:34:13.147173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0475fc-bfaa-4085-ab67-ca3cb6ddc78d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:13.162474 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0475fc-bfaa-4085-ab67-ca3cb6ddc78d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:13.167380 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99619ed9-b3b6-42a3-bc01-5a0d638dcb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:13.168009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99619ed9-b3b6-42a3-bc01-5a0d638dcb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:13.168573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99619ed9-b3b6-42a3-bc01-5a0d638dcb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:13.168857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99619ed9-b3b6-42a3-bc01-5a0d638dcb10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:13.169655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99619ed9-b3b6-42a3-bc01-5a0d638dcb10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:34:13.169959 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99619ed9-b3b6-42a3-bc01-5a0d638dcb10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:13.172249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99619ed9-b3b6-42a3-bc01-5a0d638dcb10 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:13.174178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:13.186186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1146/2327] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:34:13.191068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c2c5c-4d94-477c-84df-7a4fdb9d69e3 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:13.200767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c2c5c-4d94-477c-84df-7a4fdb9d69e3 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:13.206192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930aff23-8a10-41eb-b9cc-1106abf122d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:13.206828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aff23-8a10-41eb-b9cc-1106abf122d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:13.207256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aff23-8a10-41eb-b9cc-1106abf122d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:13.207553 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aff23-8a10-41eb-b9cc-1106abf122d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:13.208831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930aff23-8a10-41eb-b9cc-1106abf122d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:13.210233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930aff23-8a10-41eb-b9cc-1106abf122d6 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:13.375191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1182/2328] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:34:13 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:13.379501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99619ed9-b3b6-42a3-bc01-5a0d638dcb10 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:13.390155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99619ed9-b3b6-42a3-bc01-5a0d638dcb10 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:13.452107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171fadf-bb39-4a1d-a26b-6e9181f6a2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:13.453184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171fadf-bb39-4a1d-a26b-6e9181f6a2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:13.453650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171fadf-bb39-4a1d-a26b-6e9181f6a2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:13.453973 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171fadf-bb39-4a1d-a26b-6e9181f6a2c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:13.454665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171fadf-bb39-4a1d-a26b-6e9181f6a2c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:13.456658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171fadf-bb39-4a1d-a26b-6e9181f6a2c4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:13.596681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1147/2329] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:34:13 2022] POST /identity/v3/users => generated 284 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:13.601009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930aff23-8a10-41eb-b9cc-1106abf122d6 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:13.612857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930aff23-8a10-41eb-b9cc-1106abf122d6 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:13.617646 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1d67d-df14-4dfb-8c93-dec08372ce6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:13.618361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d67d-df14-4dfb-8c93-dec08372ce6f admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:13.618881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d67d-df14-4dfb-8c93-dec08372ce6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:13.619302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d67d-df14-4dfb-8c93-dec08372ce6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:13.620020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1d67d-df14-4dfb-8c93-dec08372ce6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:13.621673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1d67d-df14-4dfb-8c93-dec08372ce6f admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:13.633728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1183/2330] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:13.637475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171fadf-bb39-4a1d-a26b-6e9181f6a2c4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:13.649745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171fadf-bb39-4a1d-a26b-6e9181f6a2c4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:13.654845 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262bc52-01de-4e8c-9331-2f8a95998da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:13.656073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262bc52-01de-4e8c-9331-2f8a95998da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:13.656366 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262bc52-01de-4e8c-9331-2f8a95998da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:13.656654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262bc52-01de-4e8c-9331-2f8a95998da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:13.657432 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262bc52-01de-4e8c-9331-2f8a95998da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:13.659057 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262bc52-01de-4e8c-9331-2f8a95998da8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:13.672424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1148/2331] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:13.679885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1d67d-df14-4dfb-8c93-dec08372ce6f admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:13.691305 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1d67d-df14-4dfb-8c93-dec08372ce6f admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:13.696323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c6ce9-8391-411a-882a-5263d1e8600e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:13.697011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c6ce9-8391-411a-882a-5263d1e8600e admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:13.697500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c6ce9-8391-411a-882a-5263d1e8600e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:13.697802 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c6ce9-8391-411a-882a-5263d1e8600e admin admin] PATH_INFO: `/v3/projects/566c3102ff4b47dcb4e0f60b8c652814/users/ff2db473f6ec45b9976f9fef0174d4bd/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:13.718540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c6ce9-8391-411a-882a-5263d1e8600e admin admin] RBAC: Authorizing `identity:create_grant(project_id=566c3102ff4b47dcb4e0f60b8c652814, user_id=ff2db473f6ec45b9976f9fef0174d4bd, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:13.721081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c6ce9-8391-411a-882a-5263d1e8600e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:13.779202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1184/2332] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:13 2022] PUT /identity/v3/projects/566c3102ff4b47dcb4e0f60b8c652814/users/ff2db473f6ec45b9976f9fef0174d4bd/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:13.795785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef98a10-515b-4712-bc48-e78373fe8c6e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:13.796016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef98a10-515b-4712-bc48-e78373fe8c6e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:13.796883 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef98a10-515b-4712-bc48-e78373fe8c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:13.829110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1ef98a10-515b-4712-bc48-e78373fe8c6e None None] MFA Rules not processed for user `ff2db473f6ec45b9976f9fef0174d4bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:13.830385 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1ef98a10-515b-4712-bc48-e78373fe8c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:13.908129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1149/2333] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:13.919531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c6ce9-8391-411a-882a-5263d1e8600e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:13.922006 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7c3c6ce9-8391-411a-882a-5263d1e8600e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:13.937284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:13.997913 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c6ce9-8391-411a-882a-5263d1e8600e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:14.000055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b7ea8-41f9-46a4-94be-a7bf7e8d935e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:14.000919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b7ea8-41f9-46a4-94be-a7bf7e8d935e service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:14.001391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b7ea8-41f9-46a4-94be-a7bf7e8d935e service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:14.002575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b7ea8-41f9-46a4-94be-a7bf7e8d935e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:14.019184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b7ea8-41f9-46a4-94be-a7bf7e8d935e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:14.022134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b7ea8-41f9-46a4-94be-a7bf7e8d935e service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:14.037536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1185/2334] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:13 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:16.451669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef98a10-515b-4712-bc48-e78373fe8c6e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:16.453991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1ef98a10-515b-4712-bc48-e78373fe8c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:16.548605 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef98a10-515b-4712-bc48-e78373fe8c6e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:16.556255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:16.558028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:16.558028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:16.558028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] PATH_INFO: `/v3/users/97547d56a8684009a794a42ad1c5f489` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:16.559236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b7ea8-41f9-46a4-94be-a7bf7e8d935e service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:16.560670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97547d56a8684009a794a42ad1c5f489)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:16.560810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-141b7ea8-41f9-46a4-94be-a7bf7e8d935e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:16.563814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:16.567313 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:16.689099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b7ea8-41f9-46a4-94be-a7bf7e8d935e service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:16.698386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bdc20-5d0e-4ecc-a251-dd5f9cfedc20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:16.699285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bdc20-5d0e-4ecc-a251-dd5f9cfedc20 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:16.699736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bdc20-5d0e-4ecc-a251-dd5f9cfedc20 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:16.700291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bdc20-5d0e-4ecc-a251-dd5f9cfedc20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:16.703102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3b3bdc20-5d0e-4ecc-a251-dd5f9cfedc20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:16.720761 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bdc20-5d0e-4ecc-a251-dd5f9cfedc20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:16.722645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bdc20-5d0e-4ecc-a251-dd5f9cfedc20 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:16.733861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1186/2335] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:34:16.820623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693b78c-2b52-4a5b-8d5c-c7ecfe2b8a04 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:16.820921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693b78c-2b52-4a5b-8d5c-c7ecfe2b8a04 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:16.821205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693b78c-2b52-4a5b-8d5c-c7ecfe2b8a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:16.830269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97547d56a8684009a794a42ad1c5f489'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:16.844016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4693b78c-2b52-4a5b-8d5c-c7ecfe2b8a04 None None] MFA Rules not processed for user `61ad8186d91848bc8be42e3be51e9a75`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:16.844872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4693b78c-2b52-4a5b-8d5c-c7ecfe2b8a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:16.852421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97547d56a8684009a794a42ad1c5f489'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:16.862952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97547d56a8684009a794a42ad1c5f489'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:16.913083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1187/2336] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:16.967532 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97547d56a8684009a794a42ad1c5f489'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:16.971631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97547d56a8684009a794a42ad1c5f489'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:16.983081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1150/2337] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:34:16 2022] DELETE /identity/v3/users/97547d56a8684009a794a42ad1c5f489 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:17.261858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693b78c-2b52-4a5b-8d5c-c7ecfe2b8a04 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:17.278804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693b78c-2b52-4a5b-8d5c-c7ecfe2b8a04 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:17.337203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:17.363450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:17.364448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:17.364915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:17.365432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] PATH_INFO: `/v3/projects/f33d1845b3b643b99b066eaeee429e12` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:17.367431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33d1845b3b643b99b066eaeee429e12)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:17.370466 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:17.606270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33d1845b3b643b99b066eaeee429e12', 'request_id': 'req-54c47dd9-5035-4082-a2c3-61f72e042832'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:17.607620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:17.611543 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:17.626847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33d1845b3b643b99b066eaeee429e12', 'request_id': 'req-54c47dd9-5035-4082-a2c3-61f72e042832'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:17.699336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700b881-212e-4740-ac10-c46e27d89b57 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:17.757019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33d1845b3b643b99b066eaeee429e12', 'request_id': 'req-54c47dd9-5035-4082-a2c3-61f72e042832'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:17.773809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] The token cache is being invalidate because project f33d1845b3b643b99b066eaeee429e12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:34:17.774250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:17.775157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-54c47dd9-5035-4082-a2c3-61f72e042832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:17.777535 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1188/2338] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:34:17 2022] DELETE /identity/v3/projects/f33d1845b3b643b99b066eaeee429e12 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:17.800135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6078b3-10a0-4c45-9848-9f39fade2b6a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:17.800135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6078b3-10a0-4c45-9848-9f39fade2b6a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:17.800135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6078b3-10a0-4c45-9848-9f39fade2b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:17.804505 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c881a0-032a-44a1-bbab-cb2bcecce3d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:17.805372 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c881a0-032a-44a1-bbab-cb2bcecce3d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:17.817379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c881a0-032a-44a1-bbab-cb2bcecce3d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:17.819569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c881a0-032a-44a1-bbab-cb2bcecce3d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:17.823423 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-80c881a0-032a-44a1-bbab-cb2bcecce3d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:17.836595 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c881a0-032a-44a1-bbab-cb2bcecce3d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:17.838242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c881a0-032a-44a1-bbab-cb2bcecce3d4 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:17.850744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-9c6078b3-10a0-4c45-9848-9f39fade2b6a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:17.852416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9c6078b3-10a0-4c45-9848-9f39fade2b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:17.857241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1151/2339] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:17.985660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1189/2340] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:17.998809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20969fd0-b6e5-4a70-b14c-85712e7908ac None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:17.999944 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20969fd0-b6e5-4a70-b14c-85712e7908ac None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:18.000632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20969fd0-b6e5-4a70-b14c-85712e7908ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:18.030451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-20969fd0-b6e5-4a70-b14c-85712e7908ac None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:18.032355 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-20969fd0-b6e5-4a70-b14c-85712e7908ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:18.054434 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1152/2341] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:18.058777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6078b3-10a0-4c45-9848-9f39fade2b6a None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:18.073296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6078b3-10a0-4c45-9848-9f39fade2b6a None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:18.079069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927326c-28bc-4342-a42a-8e29188f59dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:18.079736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927326c-28bc-4342-a42a-8e29188f59dc admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:18.080021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927326c-28bc-4342-a42a-8e29188f59dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:18.080298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927326c-28bc-4342-a42a-8e29188f59dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:18.081073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927326c-28bc-4342-a42a-8e29188f59dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:34:18.081418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927326c-28bc-4342-a42a-8e29188f59dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:18.083257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927326c-28bc-4342-a42a-8e29188f59dc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:18.098376 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1190/2342] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:34:18.102900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20969fd0-b6e5-4a70-b14c-85712e7908ac None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:18.112433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20969fd0-b6e5-4a70-b14c-85712e7908ac None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:18.117200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a4dab-6890-44da-aa16-dc207f59d895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:18.117865 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a4dab-6890-44da-aa16-dc207f59d895 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:18.118155 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a4dab-6890-44da-aa16-dc207f59d895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:18.118439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a4dab-6890-44da-aa16-dc207f59d895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:18.119140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a4dab-6890-44da-aa16-dc207f59d895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:18.121164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a4dab-6890-44da-aa16-dc207f59d895 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:18.215723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1153/2343] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:34:18 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:34:18.221654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927326c-28bc-4342-a42a-8e29188f59dc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:18.231856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927326c-28bc-4342-a42a-8e29188f59dc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:18.293638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d632e9c-5350-44c1-a6d8-ae7684ae424b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:18.294273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d632e9c-5350-44c1-a6d8-ae7684ae424b admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:18.294562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d632e9c-5350-44c1-a6d8-ae7684ae424b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:18.294844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d632e9c-5350-44c1-a6d8-ae7684ae424b admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:18.295544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d632e9c-5350-44c1-a6d8-ae7684ae424b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:18.297573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d632e9c-5350-44c1-a6d8-ae7684ae424b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:18.307951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:18.432399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1191/2344] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:34:18 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:34:18.435169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a4dab-6890-44da-aa16-dc207f59d895 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:18.446709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a4dab-6890-44da-aa16-dc207f59d895 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:18.457650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241b30-50e3-4793-966f-5401c2281646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:18.459370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241b30-50e3-4793-966f-5401c2281646 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:18.459370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241b30-50e3-4793-966f-5401c2281646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:18.459370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241b30-50e3-4793-966f-5401c2281646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:18.460071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241b30-50e3-4793-966f-5401c2281646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:18.462405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241b30-50e3-4793-966f-5401c2281646 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:18.468987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:18.474628 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1154/2345] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:18.478777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d632e9c-5350-44c1-a6d8-ae7684ae424b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:18.497565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d632e9c-5350-44c1-a6d8-ae7684ae424b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:18.506016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87253886-bf3b-4bf6-a574-7966dd34f7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:18.506016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87253886-bf3b-4bf6-a574-7966dd34f7bb admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:18.506016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87253886-bf3b-4bf6-a574-7966dd34f7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:18.506016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87253886-bf3b-4bf6-a574-7966dd34f7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:18.506396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87253886-bf3b-4bf6-a574-7966dd34f7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:18.507642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87253886-bf3b-4bf6-a574-7966dd34f7bb admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:18.525393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1192/2346] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:18.529003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241b30-50e3-4793-966f-5401c2281646 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:18.537154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241b30-50e3-4793-966f-5401c2281646 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:18.548930 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222b886-cb06-4167-b794-348bf7d8f8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:18.549692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b886-cb06-4167-b794-348bf7d8f8cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:18.549983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b886-cb06-4167-b794-348bf7d8f8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:18.550266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b886-cb06-4167-b794-348bf7d8f8cf admin admin] PATH_INFO: `/v3/projects/d45f04c2af574b889ee776b6a0fa9d40/users/677cbef82d164fa8bd0a24afc4012cf7/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:18.566757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222b886-cb06-4167-b794-348bf7d8f8cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45f04c2af574b889ee776b6a0fa9d40, user_id=677cbef82d164fa8bd0a24afc4012cf7, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:18.570136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222b886-cb06-4167-b794-348bf7d8f8cf admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:18.659527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1155/2347] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:18 2022] PUT /identity/v3/projects/d45f04c2af574b889ee776b6a0fa9d40/users/677cbef82d164fa8bd0a24afc4012cf7/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:18.664871 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211dc74-fafe-4a94-bee7-5cbd17d082fa None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:18.665290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211dc74-fafe-4a94-bee7-5cbd17d082fa None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:18.665517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211dc74-fafe-4a94-bee7-5cbd17d082fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:18.687541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e211dc74-fafe-4a94-bee7-5cbd17d082fa None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:18.687541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e211dc74-fafe-4a94-bee7-5cbd17d082fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:18.752578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1193/2348] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:18.763909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222b886-cb06-4167-b794-348bf7d8f8cf admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:18.766261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4222b886-cb06-4167-b794-348bf7d8f8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:18.829207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222b886-cb06-4167-b794-348bf7d8f8cf admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:18.832978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55565c1a-a1b5-4cbc-8230-c02f11e622cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:18.833710 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565c1a-a1b5-4cbc-8230-c02f11e622cb service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:18.836168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565c1a-a1b5-4cbc-8230-c02f11e622cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:18.836168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565c1a-a1b5-4cbc-8230-c02f11e622cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:18.844983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55565c1a-a1b5-4cbc-8230-c02f11e622cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:18.846523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55565c1a-a1b5-4cbc-8230-c02f11e622cb service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:18.856484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1156/2349] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:24.626169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211dc74-fafe-4a94-bee7-5cbd17d082fa None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:24.629671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e211dc74-fafe-4a94-bee7-5cbd17d082fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:24.654319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211dc74-fafe-4a94-bee7-5cbd17d082fa None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:24.663619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:24.669916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:24.669916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:24.669916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] PATH_INFO: `/v3/users/ff2db473f6ec45b9976f9fef0174d4bd` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:24.669916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff2db473f6ec45b9976f9fef0174d4bd)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:24.669916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:25.007366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff2db473f6ec45b9976f9fef0174d4bd'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:25.041138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff2db473f6ec45b9976f9fef0174d4bd'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:25.049975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff2db473f6ec45b9976f9fef0174d4bd'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:25.059589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff2db473f6ec45b9976f9fef0174d4bd'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:25.144073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff2db473f6ec45b9976f9fef0174d4bd'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:25.156058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1194/2350] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:34:24 2022] DELETE /identity/v3/users/ff2db473f6ec45b9976f9fef0174d4bd => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:25.160123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ea5f2-499d-45d2-9642-6a47baba8ae6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:25.160739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ea5f2-499d-45d2-9642-6a47baba8ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:25.160739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ea5f2-499d-45d2-9642-6a47baba8ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:25.187145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ce1ea5f2-499d-45d2-9642-6a47baba8ae6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:25.189095 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ce1ea5f2-499d-45d2-9642-6a47baba8ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:25.317089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1157/2351] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:25.335718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:25.338450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:25.446262 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f8788-f4f3-45bb-a337-a11a8896681e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:25.455301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f8788-f4f3-45bb-a337-a11a8896681e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:25.455301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f8788-f4f3-45bb-a337-a11a8896681e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:25.472713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f3e01-3fa9-4c72-8891-0e70d5a27360 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:25.483672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e124ca-4ab3-469c-bce9-2819d53f5080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:25.484332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e124ca-4ab3-469c-bce9-2819d53f5080 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:25.484617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e124ca-4ab3-469c-bce9-2819d53f5080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:25.485090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e124ca-4ab3-469c-bce9-2819d53f5080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:25.496093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e124ca-4ab3-469c-bce9-2819d53f5080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:25.497751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e124ca-4ab3-469c-bce9-2819d53f5080 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:25.502952 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:25.511015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1195/2352] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:25.513270 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-669f8788-f4f3-45bb-a337-a11a8896681e None None] MFA Rules not processed for user `677cbef82d164fa8bd0a24afc4012cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:25.520682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-669f8788-f4f3-45bb-a337-a11a8896681e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:25.629247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1158/2353] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:25.630717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e124ca-4ab3-469c-bce9-2819d53f5080 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:25.632292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-76e124ca-4ab3-469c-bce9-2819d53f5080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:25.647058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e124ca-4ab3-469c-bce9-2819d53f5080 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:25.653125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159b7e2-e4b6-4f63-a737-b8d5baff7c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:25.656203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159b7e2-e4b6-4f63-a737-b8d5baff7c26 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:25.656203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159b7e2-e4b6-4f63-a737-b8d5baff7c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:25.656203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159b7e2-e4b6-4f63-a737-b8d5baff7c26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:25.656801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159b7e2-e4b6-4f63-a737-b8d5baff7c26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:25.659794 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159b7e2-e4b6-4f63-a737-b8d5baff7c26 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:25.781663 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1196/2354] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:34:25 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:25.785631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f8788-f4f3-45bb-a337-a11a8896681e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:25.799290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f8788-f4f3-45bb-a337-a11a8896681e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:25.806979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:25.918216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98173f9-1972-4af9-ba0e-9da4f4c14b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:25.918991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98173f9-1972-4af9-ba0e-9da4f4c14b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:25.919335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98173f9-1972-4af9-ba0e-9da4f4c14b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:25.919624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98173f9-1972-4af9-ba0e-9da4f4c14b38 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:25.920403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98173f9-1972-4af9-ba0e-9da4f4c14b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:25.931407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98173f9-1972-4af9-ba0e-9da4f4c14b38 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:26.038526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1159/2355] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:34:25 2022] POST /identity/v3/users => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:26.040329 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159b7e2-e4b6-4f63-a737-b8d5baff7c26 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:26.051246 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159b7e2-e4b6-4f63-a737-b8d5baff7c26 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:26.055961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca649d-fdbb-4901-a42f-613f5ca55ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:26.056636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca649d-fdbb-4901-a42f-613f5ca55ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.056945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca649d-fdbb-4901-a42f-613f5ca55ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.057211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca649d-fdbb-4901-a42f-613f5ca55ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.057975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca649d-fdbb-4901-a42f-613f5ca55ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:26.062104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca649d-fdbb-4901-a42f-613f5ca55ff1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:26.067972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98173f9-1972-4af9-ba0e-9da4f4c14b38 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:26.069593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f98173f9-1972-4af9-ba0e-9da4f4c14b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:26.085205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1197/2356] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:26.090384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98173f9-1972-4af9-ba0e-9da4f4c14b38 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:26.094532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca649d-fdbb-4901-a42f-613f5ca55ff1 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:26.103118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:26.104428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.104726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.105015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] PATH_INFO: `/v3/projects/566c3102ff4b47dcb4e0f60b8c652814` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.106287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=566c3102ff4b47dcb4e0f60b8c652814)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:26.109003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:26.113217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca649d-fdbb-4901-a42f-613f5ca55ff1 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:26.119100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fa3b3-f794-480e-bdfb-8fa1d9d9d25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:26.128597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fa3b3-f794-480e-bdfb-8fa1d9d9d25c admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.128597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fa3b3-f794-480e-bdfb-8fa1d9d9d25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.128597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fa3b3-f794-480e-bdfb-8fa1d9d9d25c admin admin] PATH_INFO: `/v3/projects/21203e171c8f4614b3039f5737a866ef/users/93c551395aa7400181d07af92f01d518/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.140169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fa3b3-f794-480e-bdfb-8fa1d9d9d25c admin admin] RBAC: Authorizing `identity:create_grant(project_id=21203e171c8f4614b3039f5737a866ef, user_id=93c551395aa7400181d07af92f01d518, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:26.143556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fa3b3-f794-480e-bdfb-8fa1d9d9d25c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:26.273121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1198/2357] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:26 2022] PUT /identity/v3/projects/21203e171c8f4614b3039f5737a866ef/users/93c551395aa7400181d07af92f01d518/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:26.277047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fa3b3-f794-480e-bdfb-8fa1d9d9d25c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:26.282167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '566c3102ff4b47dcb4e0f60b8c652814', 'request_id': 'req-54b1ca5e-4eac-41dd-88ff-d916920661a5'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:26.288509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '566c3102ff4b47dcb4e0f60b8c652814', 'request_id': 'req-54b1ca5e-4eac-41dd-88ff-d916920661a5'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:26.295810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '566c3102ff4b47dcb4e0f60b8c652814', 'request_id': 'req-54b1ca5e-4eac-41dd-88ff-d916920661a5'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:26.341968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fa3b3-f794-480e-bdfb-8fa1d9d9d25c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:26.412252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ea4cc-283e-48b5-9ee5-04afee99edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:26.412907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ea4cc-283e-48b5-9ee5-04afee99edae admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.413214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ea4cc-283e-48b5-9ee5-04afee99edae admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.413762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ea4cc-283e-48b5-9ee5-04afee99edae admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.414400 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ea4cc-283e-48b5-9ee5-04afee99edae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:26.415988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ea4cc-283e-48b5-9ee5-04afee99edae admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:26.426631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1199/2358] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:34:26.430642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] The token cache is being invalidate because project 566c3102ff4b47dcb4e0f60b8c652814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:34:26.430642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:26.430917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ea4cc-283e-48b5-9ee5-04afee99edae admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:26.431341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-54b1ca5e-4eac-41dd-88ff-d916920661a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:26.433249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1160/2359] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:34:26 2022] DELETE /identity/v3/projects/566c3102ff4b47dcb4e0f60b8c652814 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:34:26.438768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202eeeb-a222-4d9a-bb9d-8cb064ad14b1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.439291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202eeeb-a222-4d9a-bb9d-8cb064ad14b1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.439822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202eeeb-a222-4d9a-bb9d-8cb064ad14b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.442380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ea4cc-283e-48b5-9ee5-04afee99edae admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:26.449115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a50c37-ad9d-4954-89da-392e35afe03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:26.449956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a50c37-ad9d-4954-89da-392e35afe03c admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.450364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a50c37-ad9d-4954-89da-392e35afe03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.450786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a50c37-ad9d-4954-89da-392e35afe03c admin admin] PATH_INFO: `/v3/projects/21203e171c8f4614b3039f5737a866ef/users/93c551395aa7400181d07af92f01d518/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.454616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a50c37-ad9d-4954-89da-392e35afe03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=21203e171c8f4614b3039f5737a866ef, user_id=93c551395aa7400181d07af92f01d518, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:26.459005 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a50c37-ad9d-4954-89da-392e35afe03c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:26.465488 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b202eeeb-a222-4d9a-bb9d-8cb064ad14b1 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:26.466971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b202eeeb-a222-4d9a-bb9d-8cb064ad14b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:26.475572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1200/2360] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:26 2022] PUT /identity/v3/projects/21203e171c8f4614b3039f5737a866ef/users/93c551395aa7400181d07af92f01d518/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:26.534039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1161/2361] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:34:26.542697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0395a0-bf8f-4fb6-b2dd-acdca494e350 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.543043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0395a0-bf8f-4fb6-b2dd-acdca494e350 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.543326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0395a0-bf8f-4fb6-b2dd-acdca494e350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.565783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-3c0395a0-bf8f-4fb6-b2dd-acdca494e350 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:26.567031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3c0395a0-bf8f-4fb6-b2dd-acdca494e350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:26.577543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1201/2362] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:26.581088 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202eeeb-a222-4d9a-bb9d-8cb064ad14b1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:26.591596 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202eeeb-a222-4d9a-bb9d-8cb064ad14b1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:26.596547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb724ff-bc8d-4c09-9688-48025aacd19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:26.597208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb724ff-bc8d-4c09-9688-48025aacd19f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.597530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb724ff-bc8d-4c09-9688-48025aacd19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.597809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb724ff-bc8d-4c09-9688-48025aacd19f admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.598519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb724ff-bc8d-4c09-9688-48025aacd19f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:34:26.598829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb724ff-bc8d-4c09-9688-48025aacd19f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:26.600573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb724ff-bc8d-4c09-9688-48025aacd19f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:26.615102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1162/2363] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:34:26.624249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0395a0-bf8f-4fb6-b2dd-acdca494e350 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:26.634753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0395a0-bf8f-4fb6-b2dd-acdca494e350 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:26.641602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d5da0-7c76-41c8-bb46-db64795fd346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:26.642495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5da0-7c76-41c8-bb46-db64795fd346 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.643199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5da0-7c76-41c8-bb46-db64795fd346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.643199 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5da0-7c76-41c8-bb46-db64795fd346 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.643839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d5da0-7c76-41c8-bb46-db64795fd346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:26.645990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d5da0-7c76-41c8-bb46-db64795fd346 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:26.718165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1202/2364] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:34:26 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:26.722402 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb724ff-bc8d-4c09-9688-48025aacd19f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:26.733876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb724ff-bc8d-4c09-9688-48025aacd19f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:26.741258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:26.815641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dc396-374f-4d5a-a801-dd2b9feb2af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:26.816209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dc396-374f-4d5a-a801-dd2b9feb2af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.816498 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dc396-374f-4d5a-a801-dd2b9feb2af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.817013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dc396-374f-4d5a-a801-dd2b9feb2af8 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.817758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399dc396-374f-4d5a-a801-dd2b9feb2af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:26.820135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399dc396-374f-4d5a-a801-dd2b9feb2af8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:26.909115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1163/2365] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:34:26 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:26.910694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d5da0-7c76-41c8-bb46-db64795fd346 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:26.917058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:26.925493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d5da0-7c76-41c8-bb46-db64795fd346 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:26.947767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4d2c2-afd2-40ed-8147-167beffb3ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:26.948436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4d2c2-afd2-40ed-8147-167beffb3ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.948736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4d2c2-afd2-40ed-8147-167beffb3ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.949018 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4d2c2-afd2-40ed-8147-167beffb3ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.949791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4d2c2-afd2-40ed-8147-167beffb3ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:26.951423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4d2c2-afd2-40ed-8147-167beffb3ba2 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:26.970170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1203/2366] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:26.974610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dc396-374f-4d5a-a801-dd2b9feb2af8 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:26.992346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dc396-374f-4d5a-a801-dd2b9feb2af8 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:26.994373 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a309bdf-85d1-4bfd-a04a-acf28669b167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:26.995266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a309bdf-85d1-4bfd-a04a-acf28669b167 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:26.995649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a309bdf-85d1-4bfd-a04a-acf28669b167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:26.996197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a309bdf-85d1-4bfd-a04a-acf28669b167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:26.996988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a309bdf-85d1-4bfd-a04a-acf28669b167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:26.998772 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a309bdf-85d1-4bfd-a04a-acf28669b167 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:27.015817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1164/2367] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:34:27.016840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4d2c2-afd2-40ed-8147-167beffb3ba2 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:27.027864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4d2c2-afd2-40ed-8147-167beffb3ba2 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:27.033713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab02304-dc4d-487e-9f5c-d9bcd7504770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:27.034797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab02304-dc4d-487e-9f5c-d9bcd7504770 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:27.034797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab02304-dc4d-487e-9f5c-d9bcd7504770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:27.035139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab02304-dc4d-487e-9f5c-d9bcd7504770 admin admin] PATH_INFO: `/v3/projects/4a9e6f7f869a47439cfe6987c22b91bd/users/e0926312feee40a6bde55d1c557f1c30/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:27.052772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab02304-dc4d-487e-9f5c-d9bcd7504770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9e6f7f869a47439cfe6987c22b91bd, user_id=e0926312feee40a6bde55d1c557f1c30, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:27.055508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab02304-dc4d-487e-9f5c-d9bcd7504770 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:27.171573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1204/2368] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:27 2022] PUT /identity/v3/projects/4a9e6f7f869a47439cfe6987c22b91bd/users/e0926312feee40a6bde55d1c557f1c30/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:34:27.184599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7226c-cc95-4411-9061-45235ceadd1c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:27.188744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7226c-cc95-4411-9061-45235ceadd1c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:27.188744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7226c-cc95-4411-9061-45235ceadd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:27.222808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b1d7226c-cc95-4411-9061-45235ceadd1c None None] MFA Rules not processed for user `e0926312feee40a6bde55d1c557f1c30`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:27.224309 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b1d7226c-cc95-4411-9061-45235ceadd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:27.347461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1165/2369] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:27.351252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab02304-dc4d-487e-9f5c-d9bcd7504770 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:27.480448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab02304-dc4d-487e-9f5c-d9bcd7504770 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:27.485760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47d3be-4acc-4654-9d4a-6b8ae51fccc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:27.486967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47d3be-4acc-4654-9d4a-6b8ae51fccc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:27.486967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47d3be-4acc-4654-9d4a-6b8ae51fccc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:27.486967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47d3be-4acc-4654-9d4a-6b8ae51fccc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:27.487778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47d3be-4acc-4654-9d4a-6b8ae51fccc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:27.489796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47d3be-4acc-4654-9d4a-6b8ae51fccc7 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:27.590944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1205/2370] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:34:27 2022] POST /identity/v3/projects => generated 369 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:27.595987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7226c-cc95-4411-9061-45235ceadd1c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:27.623669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7226c-cc95-4411-9061-45235ceadd1c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:27.769435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aa979-40ce-4e73-aee0-7d1c51b50b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:27.770085 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aa979-40ce-4e73-aee0-7d1c51b50b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:27.770405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aa979-40ce-4e73-aee0-7d1c51b50b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:27.770649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aa979-40ce-4e73-aee0-7d1c51b50b8a admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:27.771939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758aa979-40ce-4e73-aee0-7d1c51b50b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:27.774173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758aa979-40ce-4e73-aee0-7d1c51b50b8a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:27.864060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1166/2371] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:34:27 2022] POST /identity/v3/users => generated 291 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:27.866476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47d3be-4acc-4654-9d4a-6b8ae51fccc7 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:27.880507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47d3be-4acc-4654-9d4a-6b8ae51fccc7 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:27.892947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bfa37-cd22-48f6-bc72-5716b7547abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:27.908805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bfa37-cd22-48f6-bc72-5716b7547abc admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:27.908805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bfa37-cd22-48f6-bc72-5716b7547abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:27.908805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bfa37-cd22-48f6-bc72-5716b7547abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:27.908805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bfa37-cd22-48f6-bc72-5716b7547abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:27.908805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bfa37-cd22-48f6-bc72-5716b7547abc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:27.912488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1206/2372] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:27.916865 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aa979-40ce-4e73-aee0-7d1c51b50b8a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:27.930687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aa979-40ce-4e73-aee0-7d1c51b50b8a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:27.941618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0686a-93a0-4026-b047-7615abb6c399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:27.943036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0686a-93a0-4026-b047-7615abb6c399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:27.943446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0686a-93a0-4026-b047-7615abb6c399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:27.943997 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0686a-93a0-4026-b047-7615abb6c399 admin admin] PATH_INFO: `/v3/projects/ce5c8fdab5cb40a89e73db996d10929a/users/e33b3eaf992c4716bef06fdba4aa6f29/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:27.949442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:27.975642 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0686a-93a0-4026-b047-7615abb6c399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce5c8fdab5cb40a89e73db996d10929a, user_id=e33b3eaf992c4716bef06fdba4aa6f29, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:27.979280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0686a-93a0-4026-b047-7615abb6c399 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:28.086586 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1167/2373] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:27 2022] PUT /identity/v3/projects/ce5c8fdab5cb40a89e73db996d10929a/users/e33b3eaf992c4716bef06fdba4aa6f29/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:34:28.089196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bfa37-cd22-48f6-bc72-5716b7547abc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:28.104768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bfa37-cd22-48f6-bc72-5716b7547abc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:28.213961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef9990-4f15-4aa8-8441-a7a30ffa155b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:28.214594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9990-4f15-4aa8-8441-a7a30ffa155b admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:28.214886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9990-4f15-4aa8-8441-a7a30ffa155b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:28.215172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9990-4f15-4aa8-8441-a7a30ffa155b admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:28.215879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef9990-4f15-4aa8-8441-a7a30ffa155b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:28.217598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef9990-4f15-4aa8-8441-a7a30ffa155b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:28.228616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1207/2374] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:28.232525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0686a-93a0-4026-b047-7615abb6c399 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:28.245055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0686a-93a0-4026-b047-7615abb6c399 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:28.251820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a529c1-e99a-47f9-8a61-5167b3bd395a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:28.252507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a529c1-e99a-47f9-8a61-5167b3bd395a admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:28.252791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a529c1-e99a-47f9-8a61-5167b3bd395a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:28.253070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a529c1-e99a-47f9-8a61-5167b3bd395a admin admin] PATH_INFO: `/v3/projects/ce5c8fdab5cb40a89e73db996d10929a/users/e33b3eaf992c4716bef06fdba4aa6f29/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:28.255105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a529c1-e99a-47f9-8a61-5167b3bd395a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce5c8fdab5cb40a89e73db996d10929a, user_id=e33b3eaf992c4716bef06fdba4aa6f29, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:28.257884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a529c1-e99a-47f9-8a61-5167b3bd395a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:28.272214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1168/2375] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:28 2022] PUT /identity/v3/projects/ce5c8fdab5cb40a89e73db996d10929a/users/e33b3eaf992c4716bef06fdba4aa6f29/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:34:28.283784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba849e-93a7-4a50-97b0-448e2ecd08d2 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:28.284228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba849e-93a7-4a50-97b0-448e2ecd08d2 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:28.284814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba849e-93a7-4a50-97b0-448e2ecd08d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:28.319297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f2ba849e-93a7-4a50-97b0-448e2ecd08d2 None None] MFA Rules not processed for user `e33b3eaf992c4716bef06fdba4aa6f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:28.320837 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f2ba849e-93a7-4a50-97b0-448e2ecd08d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:28.358253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:28.421914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1208/2376] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:28 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:28.434289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a529c1-e99a-47f9-8a61-5167b3bd395a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:28.437212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-36a529c1-e99a-47f9-8a61-5167b3bd395a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:28.591040 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a529c1-e99a-47f9-8a61-5167b3bd395a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:28.602579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85004c40-4795-49df-bf7b-f763199e166c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:28.603689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85004c40-4795-49df-bf7b-f763199e166c service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:28.604188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85004c40-4795-49df-bf7b-f763199e166c service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:28.604656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85004c40-4795-49df-bf7b-f763199e166c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:28.621500 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85004c40-4795-49df-bf7b-f763199e166c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:28.624698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85004c40-4795-49df-bf7b-f763199e166c service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:28.640842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1169/2377] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:28 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:28.784167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba849e-93a7-4a50-97b0-448e2ecd08d2 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:28.785669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f2ba849e-93a7-4a50-97b0-448e2ecd08d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:28.812355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba849e-93a7-4a50-97b0-448e2ecd08d2 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:28.819797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fd95a-5c85-43d8-a3b0-1611f352d3d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:28.822368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fd95a-5c85-43d8-a3b0-1611f352d3d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:28.822889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fd95a-5c85-43d8-a3b0-1611f352d3d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:28.824046 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fd95a-5c85-43d8-a3b0-1611f352d3d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:28.931520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fd95a-5c85-43d8-a3b0-1611f352d3d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:28.934569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fd95a-5c85-43d8-a3b0-1611f352d3d2 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:28.949049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1209/2378] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:33.354637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17640-2dfa-4b79-bde7-fddb2a87f511 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:33.356284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17640-2dfa-4b79-bde7-fddb2a87f511 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:33.356564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17640-2dfa-4b79-bde7-fddb2a87f511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:33.426523 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bdb17640-2dfa-4b79-bde7-fddb2a87f511 None None] MFA Rules not processed for user `93c551395aa7400181d07af92f01d518`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:33.430140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bdb17640-2dfa-4b79-bde7-fddb2a87f511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:33.556436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1170/2379] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:33.576420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fd95a-5c85-43d8-a3b0-1611f352d3d2 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:33.578978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c20fd95a-5c85-43d8-a3b0-1611f352d3d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:33.720379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fd95a-5c85-43d8-a3b0-1611f352d3d2 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:33.730121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f615062-4f88-4993-a87f-b7618fc25cbd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4fe04b11ec412e8f4eb67c6e54b8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:33.731030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f615062-4f88-4993-a87f-b7618fc25cbd service glance] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:33.732615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f615062-4f88-4993-a87f-b7618fc25cbd service glance] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:33.733680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f615062-4f88-4993-a87f-b7618fc25cbd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:33.737347 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9f615062-4f88-4993-a87f-b7618fc25cbd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:33.864555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f615062-4f88-4993-a87f-b7618fc25cbd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:33.866173 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f615062-4f88-4993-a87f-b7618fc25cbd service glance] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:33.878290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1210/2380] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:34.197924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce502e30-7385-4ccc-92fc-14238063c6fc None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:34.198470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce502e30-7385-4ccc-92fc-14238063c6fc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:34.198893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce502e30-7385-4ccc-92fc-14238063c6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:34.224645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ce502e30-7385-4ccc-92fc-14238063c6fc None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:34.225904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ce502e30-7385-4ccc-92fc-14238063c6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:34.293020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1171/2381] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:34:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:34.439320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f615062-4f88-4993-a87f-b7618fc25cbd service glance] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:34.441069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9f615062-4f88-4993-a87f-b7618fc25cbd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:34.461343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:34.516251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f615062-4f88-4993-a87f-b7618fc25cbd service glance] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:34.522570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4abfd-17bf-46d0-9a54-10bd31b4d68e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:34.523288 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4abfd-17bf-46d0-9a54-10bd31b4d68e service swift] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:34.523645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4abfd-17bf-46d0-9a54-10bd31b4d68e service swift] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:34.523993 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4abfd-17bf-46d0-9a54-10bd31b4d68e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:34.539776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4abfd-17bf-46d0-9a54-10bd31b4d68e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:34.541501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4abfd-17bf-46d0-9a54-10bd31b4d68e service swift] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:34.551670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1211/2382] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:34:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:34:34.559437 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce502e30-7385-4ccc-92fc-14238063c6fc None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:34.567985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:34.678775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce502e30-7385-4ccc-92fc-14238063c6fc None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:34.685289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:34.690187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:34.690689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:34.691185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] PATH_INFO: `/v3/users/e0926312feee40a6bde55d1c557f1c30` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:34.692768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0926312feee40a6bde55d1c557f1c30)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:34.698716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:34.959170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0926312feee40a6bde55d1c557f1c30'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:34.984227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0926312feee40a6bde55d1c557f1c30'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:34.993929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0926312feee40a6bde55d1c557f1c30'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:35.069778 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0926312feee40a6bde55d1c557f1c30'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:35.071669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0926312feee40a6bde55d1c557f1c30'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:35.082236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1172/2383] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:34:34 2022] DELETE /identity/v3/users/e0926312feee40a6bde55d1c557f1c30 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:35.086669 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123dec-2bc6-459d-a4c0-a99b41d3d297 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:35.087162 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123dec-2bc6-459d-a4c0-a99b41d3d297 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:35.087280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123dec-2bc6-459d-a4c0-a99b41d3d297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:35.138303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-c6123dec-2bc6-459d-a4c0-a99b41d3d297 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:35.142769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c6123dec-2bc6-459d-a4c0-a99b41d3d297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:35.271207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1212/2384] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:35.289586 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:35.291044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:35.417259 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e4306-78bc-415e-9883-381ac0db918f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:35.428030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b4b90-419b-446d-8a89-ac30420306fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:35.428694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b4b90-419b-446d-8a89-ac30420306fb service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:35.428987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b4b90-419b-446d-8a89-ac30420306fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:35.429392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b4b90-419b-446d-8a89-ac30420306fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:35.448795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b4b90-419b-446d-8a89-ac30420306fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:35.450547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b4b90-419b-446d-8a89-ac30420306fb service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:35.481413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1173/2385] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:36.335674 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6123dec-2bc6-459d-a4c0-a99b41d3d297 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:36.337223 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c6123dec-2bc6-459d-a4c0-a99b41d3d297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:36.391099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6123dec-2bc6-459d-a4c0-a99b41d3d297 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:36.401410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:36.402385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:36.403116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:36.403869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] PATH_INFO: `/v3/projects/4a9e6f7f869a47439cfe6987c22b91bd` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:36.412077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9e6f7f869a47439cfe6987c22b91bd)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:36.414795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:36.632090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9e6f7f869a47439cfe6987c22b91bd', 'request_id': 'req-7c385ede-aba6-4dfe-9f05-83cda76e93d6'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:36.644860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9e6f7f869a47439cfe6987c22b91bd', 'request_id': 'req-7c385ede-aba6-4dfe-9f05-83cda76e93d6'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:36.726844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9e6f7f869a47439cfe6987c22b91bd', 'request_id': 'req-7c385ede-aba6-4dfe-9f05-83cda76e93d6'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:36.741198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] The token cache is being invalidate because project 4a9e6f7f869a47439cfe6987c22b91bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:34:36.741513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:36.742549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:36.746310 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1213/2386] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:34:36 2022] DELETE /identity/v3/projects/4a9e6f7f869a47439cfe6987c22b91bd => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:36.751772 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b4b90-419b-446d-8a89-ac30420306fb service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:36.774991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b4b90-419b-446d-8a89-ac30420306fb service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:36.812096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:36.892382 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:36.893072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:36.893392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:36.893678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] PATH_INFO: `/v3/users/e33b3eaf992c4716bef06fdba4aa6f29` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:36.897241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e33b3eaf992c4716bef06fdba4aa6f29)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:36.900283 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:37.070376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e33b3eaf992c4716bef06fdba4aa6f29'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:37.096146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e33b3eaf992c4716bef06fdba4aa6f29'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:37.105364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e33b3eaf992c4716bef06fdba4aa6f29'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:37.184972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e33b3eaf992c4716bef06fdba4aa6f29'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:37.186809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf7472c2-df2d-4a23-8ed5-f9fd307b03c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e33b3eaf992c4716bef06fdba4aa6f29'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:37.197557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1174/2387] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:34:36 2022] DELETE /identity/v3/users/e33b3eaf992c4716bef06fdba4aa6f29 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:37.833785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:37.837627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:37.863079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c385ede-aba6-4dfe-9f05-83cda76e93d6 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:37.914159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:37.951676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:37.953132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:37.953614 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:37.954303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] PATH_INFO: `/v3/projects/ce5c8fdab5cb40a89e73db996d10929a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:37.955773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce5c8fdab5cb40a89e73db996d10929a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:37.958527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:38.130436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce5c8fdab5cb40a89e73db996d10929a', 'request_id': 'req-b61c8755-6d15-40ca-8362-5e5c6568051c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:38.154335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce5c8fdab5cb40a89e73db996d10929a', 'request_id': 'req-b61c8755-6d15-40ca-8362-5e5c6568051c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:38.481145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce5c8fdab5cb40a89e73db996d10929a', 'request_id': 'req-b61c8755-6d15-40ca-8362-5e5c6568051c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:38.494180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] The token cache is being invalidate because project ce5c8fdab5cb40a89e73db996d10929a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:34:38.494472 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:38.495121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b61c8755-6d15-40ca-8362-5e5c6568051c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:38.496695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1214/2388] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:34:37 2022] DELETE /identity/v3/projects/ce5c8fdab5cb40a89e73db996d10929a => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:38.502741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6406d2-ef4b-4a18-af42-cae8f310fc2f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:38.508597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6406d2-ef4b-4a18-af42-cae8f310fc2f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:38.508597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6406d2-ef4b-4a18-af42-cae8f310fc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:38.525958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0c6406d2-ef4b-4a18-af42-cae8f310fc2f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:38.525958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0c6406d2-ef4b-4a18-af42-cae8f310fc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:38.627549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1175/2389] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:34:38.635407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9a222-e831-47a2-97f4-01cd901a9ab0 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:38.636307 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9a222-e831-47a2-97f4-01cd901a9ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:38.636973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9a222-e831-47a2-97f4-01cd901a9ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:38.683152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-d3d9a222-e831-47a2-97f4-01cd901a9ab0 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:38.684399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d3d9a222-e831-47a2-97f4-01cd901a9ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:38.717396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1215/2390] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:38.718702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6406d2-ef4b-4a18-af42-cae8f310fc2f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:38.729374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6406d2-ef4b-4a18-af42-cae8f310fc2f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:38.751685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad29ec7-41dc-4c8f-890e-81b4f53891ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:38.752375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad29ec7-41dc-4c8f-890e-81b4f53891ee admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:38.752668 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad29ec7-41dc-4c8f-890e-81b4f53891ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:38.752955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad29ec7-41dc-4c8f-890e-81b4f53891ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:38.758904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad29ec7-41dc-4c8f-890e-81b4f53891ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:34:38.758904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad29ec7-41dc-4c8f-890e-81b4f53891ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:38.760555 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad29ec7-41dc-4c8f-890e-81b4f53891ee admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:38.797464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1176/2391] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:34:38.798964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9a222-e831-47a2-97f4-01cd901a9ab0 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:38.818577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9a222-e831-47a2-97f4-01cd901a9ab0 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:38.832511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5f9b8-4d39-485c-88e9-3a770efd96fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:38.833166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5f9b8-4d39-485c-88e9-3a770efd96fc admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:38.833482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5f9b8-4d39-485c-88e9-3a770efd96fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:38.833769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5f9b8-4d39-485c-88e9-3a770efd96fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:38.834967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5f9b8-4d39-485c-88e9-3a770efd96fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:38.851994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5f9b8-4d39-485c-88e9-3a770efd96fc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:38.968174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1216/2392] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:34:38 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:38.970626 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad29ec7-41dc-4c8f-890e-81b4f53891ee admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:38.984686 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad29ec7-41dc-4c8f-890e-81b4f53891ee admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:39.056925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:39.128862 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dab7ce-bc1d-4eb4-ad9e-e54e19964505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:39.129575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dab7ce-bc1d-4eb4-ad9e-e54e19964505 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:39.129862 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dab7ce-bc1d-4eb4-ad9e-e54e19964505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:39.130144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dab7ce-bc1d-4eb4-ad9e-e54e19964505 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:39.130823 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dab7ce-bc1d-4eb4-ad9e-e54e19964505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:39.132797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dab7ce-bc1d-4eb4-ad9e-e54e19964505 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:39.299199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1177/2393] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:34:38 2022] POST /identity/v3/users => generated 294 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:39.304322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5f9b8-4d39-485c-88e9-3a770efd96fc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:39.329542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fc571-6cb2-483e-827e-7a3f45ea665e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:39.329881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fc571-6cb2-483e-827e-7a3f45ea665e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:39.329996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5f9b8-4d39-485c-88e9-3a770efd96fc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:39.330164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fc571-6cb2-483e-827e-7a3f45ea665e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:39.343725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe76c7-f46c-4792-be8e-e9be5065abdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:39.344432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe76c7-f46c-4792-be8e-e9be5065abdd admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:39.344729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe76c7-f46c-4792-be8e-e9be5065abdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:39.345081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe76c7-f46c-4792-be8e-e9be5065abdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:39.345978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe76c7-f46c-4792-be8e-e9be5065abdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:39.347642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe76c7-f46c-4792-be8e-e9be5065abdd admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:39.382797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-577fc571-6cb2-483e-827e-7a3f45ea665e None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:39.384067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-577fc571-6cb2-483e-827e-7a3f45ea665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:39.395870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1217/2394] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:39.396876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe76c7-f46c-4792-be8e-e9be5065abdd admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:39.415345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe76c7-f46c-4792-be8e-e9be5065abdd admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:39.431166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a98a-7296-471e-8ba8-71890d4f9537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:39.432192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a98a-7296-471e-8ba8-71890d4f9537 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:39.439331 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a98a-7296-471e-8ba8-71890d4f9537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:39.440495 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a98a-7296-471e-8ba8-71890d4f9537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:39.443417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a98a-7296-471e-8ba8-71890d4f9537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:39.463365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a98a-7296-471e-8ba8-71890d4f9537 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:39.489147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1218/2395] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:39.495901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a98a-7296-471e-8ba8-71890d4f9537 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:39.519719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a98a-7296-471e-8ba8-71890d4f9537 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:39.534324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1178/2396] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:34:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:39.534886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6e712-5aee-46e3-876c-76dac44cb60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:39.534886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6e712-5aee-46e3-876c-76dac44cb60e admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:39.535289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6e712-5aee-46e3-876c-76dac44cb60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:39.535804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6e712-5aee-46e3-876c-76dac44cb60e admin admin] PATH_INFO: `/v3/projects/58151a361f6b493da294841877da542a/users/39777219aac24d53927c8068099d1c85/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:39.573128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6e712-5aee-46e3-876c-76dac44cb60e admin admin] RBAC: Authorizing `identity:create_grant(project_id=58151a361f6b493da294841877da542a, user_id=39777219aac24d53927c8068099d1c85, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:39.574303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fc571-6cb2-483e-827e-7a3f45ea665e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:39.576059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-577fc571-6cb2-483e-827e-7a3f45ea665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:39.576577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6e712-5aee-46e3-876c-76dac44cb60e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:39.664699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1219/2397] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:39 2022] PUT /identity/v3/projects/58151a361f6b493da294841877da542a/users/39777219aac24d53927c8068099d1c85/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:39.667172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae21d5-6ae2-4111-8ffa-2d4768b40e17 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:39.667566 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae21d5-6ae2-4111-8ffa-2d4768b40e17 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:39.668009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae21d5-6ae2-4111-8ffa-2d4768b40e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:39.727684 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f9ae21d5-6ae2-4111-8ffa-2d4768b40e17 None None] MFA Rules not processed for user `39777219aac24d53927c8068099d1c85`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:39.728904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f9ae21d5-6ae2-4111-8ffa-2d4768b40e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:39.750613 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fc571-6cb2-483e-827e-7a3f45ea665e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:39.814550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:39.859931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1220/2398] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:39.868605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae21d5-6ae2-4111-8ffa-2d4768b40e17 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:39.907911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e3445-bb65-4a48-9fd1-c3d48c23a5db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:39.908721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e3445-bb65-4a48-9fd1-c3d48c23a5db service swift] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:39.909006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e3445-bb65-4a48-9fd1-c3d48c23a5db service swift] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:39.909451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e3445-bb65-4a48-9fd1-c3d48c23a5db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:39.929074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:39.953992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae21d5-6ae2-4111-8ffa-2d4768b40e17 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:39.962628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512550c4-282a-4aad-b93a-4adfa6693e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:39.963477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512550c4-282a-4aad-b93a-4adfa6693e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:39.963767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512550c4-282a-4aad-b93a-4adfa6693e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:39.964051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512550c4-282a-4aad-b93a-4adfa6693e7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:39.964861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512550c4-282a-4aad-b93a-4adfa6693e7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:39.976287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512550c4-282a-4aad-b93a-4adfa6693e7e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:40.021951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e3445-bb65-4a48-9fd1-c3d48c23a5db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:40.023815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e3445-bb65-4a48-9fd1-c3d48c23a5db service swift] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:40.035174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1179/2399] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:34:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:34:40.109769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e3445-bb65-4a48-9fd1-c3d48c23a5db service swift] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:40.114951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1221/2400] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:34:39 2022] POST /identity/v3/projects => generated 387 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:40.124301 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e3445-bb65-4a48-9fd1-c3d48c23a5db service swift] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:40.221188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d6fa1-52b4-453d-b097-61e4438c3c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:40.222008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d6fa1-52b4-453d-b097-61e4438c3c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:40.222417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d6fa1-52b4-453d-b097-61e4438c3c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:40.222802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d6fa1-52b4-453d-b097-61e4438c3c87 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:40.223946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d6fa1-52b4-453d-b097-61e4438c3c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:40.226538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d6fa1-52b4-453d-b097-61e4438c3c87 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:40.317805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1180/2401] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:34:40 2022] POST /identity/v3/users => generated 300 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:34:40.327303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512550c4-282a-4aad-b93a-4adfa6693e7e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:40.340188 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512550c4-282a-4aad-b93a-4adfa6693e7e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:40.362204 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f0c3b-96fe-44c7-aa16-c498a17a045a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:40.362936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f0c3b-96fe-44c7-aa16-c498a17a045a admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:40.363230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f0c3b-96fe-44c7-aa16-c498a17a045a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:40.364136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f0c3b-96fe-44c7-aa16-c498a17a045a admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:40.364888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f0c3b-96fe-44c7-aa16-c498a17a045a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:40.370385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f0c3b-96fe-44c7-aa16-c498a17a045a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:40.396270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1222/2402] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:40.400295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d6fa1-52b4-453d-b097-61e4438c3c87 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:40.437149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d6fa1-52b4-453d-b097-61e4438c3c87 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:40.453096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d9c7-b358-4f4d-81f0-d8ad064954ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:40.454118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d9c7-b358-4f4d-81f0-d8ad064954ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:40.454590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d9c7-b358-4f4d-81f0-d8ad064954ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:40.455277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d9c7-b358-4f4d-81f0-d8ad064954ee admin admin] PATH_INFO: `/v3/projects/74a043a871a24c2092600ea07eab3bd4/users/27fd7583f6dc474f8c8e7f5b0eb45c0e/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:40.501368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02d9c7-b358-4f4d-81f0-d8ad064954ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a043a871a24c2092600ea07eab3bd4, user_id=27fd7583f6dc474f8c8e7f5b0eb45c0e, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:40.504392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02d9c7-b358-4f4d-81f0-d8ad064954ee admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:40.600906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1181/2403] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:40 2022] PUT /identity/v3/projects/74a043a871a24c2092600ea07eab3bd4/users/27fd7583f6dc474f8c8e7f5b0eb45c0e/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:40.604975 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f0c3b-96fe-44c7-aa16-c498a17a045a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:40.621155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f0c3b-96fe-44c7-aa16-c498a17a045a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:40.783602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcdced-e9e4-4657-9a3a-ac75eb860fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:40.784408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcdced-e9e4-4657-9a3a-ac75eb860fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:40.785855 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcdced-e9e4-4657-9a3a-ac75eb860fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:40.786212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcdced-e9e4-4657-9a3a-ac75eb860fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:40.787175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dcdced-e9e4-4657-9a3a-ac75eb860fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:40.792347 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dcdced-e9e4-4657-9a3a-ac75eb860fdd admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:40.829185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1223/2404] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:40.831877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d9c7-b358-4f4d-81f0-d8ad064954ee admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:40.850475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d9c7-b358-4f4d-81f0-d8ad064954ee admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:40.867840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591d703-d511-4a47-bbb7-4ac57bba77e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:40.868625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591d703-d511-4a47-bbb7-4ac57bba77e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:40.869009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591d703-d511-4a47-bbb7-4ac57bba77e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:40.869412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591d703-d511-4a47-bbb7-4ac57bba77e3 admin admin] PATH_INFO: `/v3/projects/74a043a871a24c2092600ea07eab3bd4/users/27fd7583f6dc474f8c8e7f5b0eb45c0e/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:40.871816 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591d703-d511-4a47-bbb7-4ac57bba77e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a043a871a24c2092600ea07eab3bd4, user_id=27fd7583f6dc474f8c8e7f5b0eb45c0e, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:40.877540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591d703-d511-4a47-bbb7-4ac57bba77e3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:40.906516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1182/2405] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:40 2022] PUT /identity/v3/projects/74a043a871a24c2092600ea07eab3bd4/users/27fd7583f6dc474f8c8e7f5b0eb45c0e/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:34:40.916603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e5589-1922-4378-a407-ad2c52f57cd3 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:40.917159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e5589-1922-4378-a407-ad2c52f57cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:40.917427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e5589-1922-4378-a407-ad2c52f57cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:40.981070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-d48e5589-1922-4378-a407-ad2c52f57cd3 None None] MFA Rules not processed for user `27fd7583f6dc474f8c8e7f5b0eb45c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:40.982494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d48e5589-1922-4378-a407-ad2c52f57cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:41.071757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:41.171776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1224/2406] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:41.185893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591d703-d511-4a47-bbb7-4ac57bba77e3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:41.188385 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8591d703-d511-4a47-bbb7-4ac57bba77e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:41.334025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591d703-d511-4a47-bbb7-4ac57bba77e3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:41.346292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d3dcd-e774-4e57-ab68-eefcd414023e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:41.347438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d3dcd-e774-4e57-ab68-eefcd414023e service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:41.352869 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d3dcd-e774-4e57-ab68-eefcd414023e service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:41.353384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d3dcd-e774-4e57-ab68-eefcd414023e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:41.375425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d3dcd-e774-4e57-ab68-eefcd414023e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:41.377124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d3dcd-e774-4e57-ab68-eefcd414023e service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:41.389718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1183/2407] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:41.545366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e5589-1922-4378-a407-ad2c52f57cd3 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:41.728346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e5589-1922-4378-a407-ad2c52f57cd3 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:41.740332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:41.741396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:41.741861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:41.742333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] PATH_INFO: `/v3/users/39777219aac24d53927c8068099d1c85` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:41.744747 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39777219aac24d53927c8068099d1c85)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:41.755820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:42.017686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39777219aac24d53927c8068099d1c85'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:42.049800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39777219aac24d53927c8068099d1c85'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:42.053863 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39777219aac24d53927c8068099d1c85'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:42.071281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39777219aac24d53927c8068099d1c85'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:42.170895 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39777219aac24d53927c8068099d1c85'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:42.186142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1225/2408] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:34:41 2022] DELETE /identity/v3/users/39777219aac24d53927c8068099d1c85 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:42.187603 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02fc66-908d-4c56-beb7-2bf6e4defa99 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:42.188094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02fc66-908d-4c56-beb7-2bf6e4defa99 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:42.188357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02fc66-908d-4c56-beb7-2bf6e4defa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:42.196962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:42.213158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8b02fc66-908d-4c56-beb7-2bf6e4defa99 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:42.214558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8b02fc66-908d-4c56-beb7-2bf6e4defa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:42.308573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1184/2409] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:34:42.318843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:42.320636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:42.402272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:42.418165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36d260-8136-4c81-80b6-1aa6e0f41305 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:42.432717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5232f-c73d-4ba8-a300-b2a352625372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:42.433341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5232f-c73d-4ba8-a300-b2a352625372 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:42.433696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5232f-c73d-4ba8-a300-b2a352625372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:42.434381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5232f-c73d-4ba8-a300-b2a352625372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:42.447126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5232f-c73d-4ba8-a300-b2a352625372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:42.454866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5232f-c73d-4ba8-a300-b2a352625372 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:42.482296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1226/2410] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:43.290066 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02fc66-908d-4c56-beb7-2bf6e4defa99 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:43.299546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8b02fc66-908d-4c56-beb7-2bf6e4defa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:43.306020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02fc66-908d-4c56-beb7-2bf6e4defa99 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:43.313026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:43.313884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:43.314169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:43.314454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] PATH_INFO: `/v3/projects/58151a361f6b493da294841877da542a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:43.318144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=58151a361f6b493da294841877da542a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:43.320220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:43.477967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58151a361f6b493da294841877da542a', 'request_id': 'req-7442948d-e336-4cd5-9e93-59f11a400b0a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:43.489065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58151a361f6b493da294841877da542a', 'request_id': 'req-7442948d-e336-4cd5-9e93-59f11a400b0a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:43.500058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58151a361f6b493da294841877da542a', 'request_id': 'req-7442948d-e336-4cd5-9e93-59f11a400b0a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:43.836874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] The token cache is being invalidate because project 58151a361f6b493da294841877da542a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:34:43.837166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:43.837893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:43.840637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1185/2411] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:34:43 2022] DELETE /identity/v3/projects/58151a361f6b493da294841877da542a => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:43.847915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5232f-c73d-4ba8-a300-b2a352625372 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:43.867587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5232f-c73d-4ba8-a300-b2a352625372 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:43.942760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:43.943729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:43.944194 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:43.944648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] PATH_INFO: `/v3/users/27fd7583f6dc474f8c8e7f5b0eb45c0e` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:43.946043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27fd7583f6dc474f8c8e7f5b0eb45c0e)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:43.948539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:44.241059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27fd7583f6dc474f8c8e7f5b0eb45c0e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:44.262387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27fd7583f6dc474f8c8e7f5b0eb45c0e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:44.265278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27fd7583f6dc474f8c8e7f5b0eb45c0e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:44.274137 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27fd7583f6dc474f8c8e7f5b0eb45c0e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:44.408581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-f39082bf-0de0-4609-950e-6911f5910894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27fd7583f6dc474f8c8e7f5b0eb45c0e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:44.416623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1227/2412] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:34:43 2022] DELETE /identity/v3/users/27fd7583f6dc474f8c8e7f5b0eb45c0e => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:45.043935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:45.046007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:45.088643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442948d-e336-4cd5-9e93-59f11a400b0a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:45.194654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:45.195599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:45.196065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:45.196520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] PATH_INFO: `/v3/projects/74a043a871a24c2092600ea07eab3bd4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:45.204831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a043a871a24c2092600ea07eab3bd4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:45.207117 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:45.355902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:45.384017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a043a871a24c2092600ea07eab3bd4', 'request_id': 'req-4138f7ce-2a74-4130-a47f-a00a138ff841'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:45.396475 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a043a871a24c2092600ea07eab3bd4', 'request_id': 'req-4138f7ce-2a74-4130-a47f-a00a138ff841'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:45.422491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a043a871a24c2092600ea07eab3bd4', 'request_id': 'req-4138f7ce-2a74-4130-a47f-a00a138ff841'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:45.505250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] The token cache is being invalidate because project 74a043a871a24c2092600ea07eab3bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:34:45.508412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:45.510872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-4138f7ce-2a74-4130-a47f-a00a138ff841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:45.515426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1186/2413] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:34:45 2022] DELETE /identity/v3/projects/74a043a871a24c2092600ea07eab3bd4 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:45.520117 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa181-5c09-4ee3-8074-04531546d05b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:45.520585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa181-5c09-4ee3-8074-04531546d05b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:45.520680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa181-5c09-4ee3-8074-04531546d05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:45.561503 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-bdeaa181-5c09-4ee3-8074-04531546d05b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:45.562893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-bdeaa181-5c09-4ee3-8074-04531546d05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:45.696808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1228/2414] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:45.702251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486e3d4-6db8-4c18-90af-5941eaee5350 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:45.702546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486e3d4-6db8-4c18-90af-5941eaee5350 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:45.703245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486e3d4-6db8-4c18-90af-5941eaee5350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:45.736242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c486e3d4-6db8-4c18-90af-5941eaee5350 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:45.740764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c486e3d4-6db8-4c18-90af-5941eaee5350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:45.757183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1187/2415] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:45.761689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaa181-5c09-4ee3-8074-04531546d05b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:45.778168 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaa181-5c09-4ee3-8074-04531546d05b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:45.796328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c8ccb-4b6f-4e83-a608-a547921dfa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:45.797028 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c8ccb-4b6f-4e83-a608-a547921dfa7d admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:45.797692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c8ccb-4b6f-4e83-a608-a547921dfa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:45.797989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c8ccb-4b6f-4e83-a608-a547921dfa7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:45.798717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c8ccb-4b6f-4e83-a608-a547921dfa7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:34:45.799139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c8ccb-4b6f-4e83-a608-a547921dfa7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:45.801158 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c8ccb-4b6f-4e83-a608-a547921dfa7d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:45.808995 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:45.824054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1229/2416] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:34:45.829428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486e3d4-6db8-4c18-90af-5941eaee5350 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:45.841704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486e3d4-6db8-4c18-90af-5941eaee5350 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:45.848921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c3f33-c577-40d5-98ed-fdbf66d2e80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:45.849705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c3f33-c577-40d5-98ed-fdbf66d2e80a admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:45.850200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c3f33-c577-40d5-98ed-fdbf66d2e80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:45.850648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c3f33-c577-40d5-98ed-fdbf66d2e80a admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:45.851417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c3f33-c577-40d5-98ed-fdbf66d2e80a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:45.854075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c3f33-c577-40d5-98ed-fdbf66d2e80a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:45.974971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1188/2417] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:34:45 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:34:45.978814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c8ccb-4b6f-4e83-a608-a547921dfa7d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:45.994543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c8ccb-4b6f-4e83-a608-a547921dfa7d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:46.086804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34902c4-1d9b-45da-bf8f-62294c9f51f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:46.087812 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34902c4-1d9b-45da-bf8f-62294c9f51f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:46.088120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34902c4-1d9b-45da-bf8f-62294c9f51f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:46.088397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34902c4-1d9b-45da-bf8f-62294c9f51f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:46.089096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34902c4-1d9b-45da-bf8f-62294c9f51f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:46.091522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34902c4-1d9b-45da-bf8f-62294c9f51f8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:46.191652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1230/2418] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:34:45 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:46.195561 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c3f33-c577-40d5-98ed-fdbf66d2e80a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:46.209790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c3f33-c577-40d5-98ed-fdbf66d2e80a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:46.215684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac882a6-5481-464f-92ca-48a631c5e986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:46.216741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac882a6-5481-464f-92ca-48a631c5e986 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:46.217352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac882a6-5481-464f-92ca-48a631c5e986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:46.217960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac882a6-5481-464f-92ca-48a631c5e986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:46.219147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac882a6-5481-464f-92ca-48a631c5e986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:46.221028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac882a6-5481-464f-92ca-48a631c5e986 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:46.233464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1189/2419] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:46.249692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34902c4-1d9b-45da-bf8f-62294c9f51f8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:46.265932 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34902c4-1d9b-45da-bf8f-62294c9f51f8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:46.278263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ba5a6-7029-4d5f-a29b-d8135043a3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:46.279430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ba5a6-7029-4d5f-a29b-d8135043a3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:46.280380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ba5a6-7029-4d5f-a29b-d8135043a3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:46.280901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ba5a6-7029-4d5f-a29b-d8135043a3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:46.281868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ba5a6-7029-4d5f-a29b-d8135043a3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:46.283830 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ba5a6-7029-4d5f-a29b-d8135043a3b7 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:46.298510 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1231/2420] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:46.303091 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac882a6-5481-464f-92ca-48a631c5e986 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:46.316149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac882a6-5481-464f-92ca-48a631c5e986 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:46.322043 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a713dc8-42ce-4379-b864-18dfd01537e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:46.322947 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a713dc8-42ce-4379-b864-18dfd01537e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:46.323333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a713dc8-42ce-4379-b864-18dfd01537e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:46.323723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a713dc8-42ce-4379-b864-18dfd01537e6 admin admin] PATH_INFO: `/v3/projects/426c38bb690049b99a4d39c8f96f41e5/users/06d63a8b7fc34c3eb7ba50c8dba4d78c/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:46.342768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a713dc8-42ce-4379-b864-18dfd01537e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=426c38bb690049b99a4d39c8f96f41e5, user_id=06d63a8b7fc34c3eb7ba50c8dba4d78c, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:46.350602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a713dc8-42ce-4379-b864-18dfd01537e6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:46.469349 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1190/2421] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:46 2022] PUT /identity/v3/projects/426c38bb690049b99a4d39c8f96f41e5/users/06d63a8b7fc34c3eb7ba50c8dba4d78c/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:34:46.481174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606a4db-e66f-4200-a8d1-173ef64a97e8 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:46.481641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606a4db-e66f-4200-a8d1-173ef64a97e8 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:46.482050 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606a4db-e66f-4200-a8d1-173ef64a97e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:46.522002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-c606a4db-e66f-4200-a8d1-173ef64a97e8 None None] MFA Rules not processed for user `06d63a8b7fc34c3eb7ba50c8dba4d78c`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:46.524457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c606a4db-e66f-4200-a8d1-173ef64a97e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:46.627625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1232/2422] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:46.884225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a713dc8-42ce-4379-b864-18dfd01537e6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:46.884225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5a713dc8-42ce-4379-b864-18dfd01537e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:46.884225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a713dc8-42ce-4379-b864-18dfd01537e6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:46.884622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff7914-2dbe-425f-945b-56326e2ca4d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:46.884622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff7914-2dbe-425f-945b-56326e2ca4d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:46.884622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff7914-2dbe-425f-945b-56326e2ca4d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:46.884622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff7914-2dbe-425f-945b-56326e2ca4d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:46.884919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff7914-2dbe-425f-945b-56326e2ca4d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:46.884919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff7914-2dbe-425f-945b-56326e2ca4d4 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:46.884919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1191/2423] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:46.885138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606a4db-e66f-4200-a8d1-173ef64a97e8 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:46.900890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606a4db-e66f-4200-a8d1-173ef64a97e8 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:46.906988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:46.907666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:46.908123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:46.908412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] PATH_INFO: `/v3/users/06d63a8b7fc34c3eb7ba50c8dba4d78c` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:46.909680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d63a8b7fc34c3eb7ba50c8dba4d78c)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:46.911806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:46.933277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:47.179904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d63a8b7fc34c3eb7ba50c8dba4d78c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:47.201715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d63a8b7fc34c3eb7ba50c8dba4d78c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:47.202632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d63a8b7fc34c3eb7ba50c8dba4d78c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:47.218129 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d63a8b7fc34c3eb7ba50c8dba4d78c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:47.299287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d63a8b7fc34c3eb7ba50c8dba4d78c'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:47.306765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1233/2424] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:34:46 2022] DELETE /identity/v3/users/06d63a8b7fc34c3eb7ba50c8dba4d78c => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:47.323687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540560b1-cd8a-4183-a603-f76d8a2756bc None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:47.323687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540560b1-cd8a-4183-a603-f76d8a2756bc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:47.323687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540560b1-cd8a-4183-a603-f76d8a2756bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:47.351768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-540560b1-cd8a-4183-a603-f76d8a2756bc None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:47.354510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-540560b1-cd8a-4183-a603-f76d8a2756bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:47.449910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:47.470083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1192/2425] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:47.477146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:47.479485 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:47.608355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf03f8-12a0-4087-a857-7f4e367b9fe8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:47.614035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ded29-c18b-47f5-a4c9-f1c9b84ef664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:47.614813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ded29-c18b-47f5-a4c9-f1c9b84ef664 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:47.615214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ded29-c18b-47f5-a4c9-f1c9b84ef664 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:47.615616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ded29-c18b-47f5-a4c9-f1c9b84ef664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:47.627316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ded29-c18b-47f5-a4c9-f1c9b84ef664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:47.655845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ded29-c18b-47f5-a4c9-f1c9b84ef664 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:47.655845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1234/2426] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:48.633411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540560b1-cd8a-4183-a603-f76d8a2756bc None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:48.635984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-540560b1-cd8a-4183-a603-f76d8a2756bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:48.663126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540560b1-cd8a-4183-a603-f76d8a2756bc None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:48.672102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:48.672818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:48.673295 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:48.673735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] PATH_INFO: `/v3/projects/426c38bb690049b99a4d39c8f96f41e5` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:48.675151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=426c38bb690049b99a4d39c8f96f41e5)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:48.677254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:48.806127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '426c38bb690049b99a4d39c8f96f41e5', 'request_id': 'req-a7832f25-c002-48b8-b2fc-7fe814a563ba'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:48.815969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '426c38bb690049b99a4d39c8f96f41e5', 'request_id': 'req-a7832f25-c002-48b8-b2fc-7fe814a563ba'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:48.824516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '426c38bb690049b99a4d39c8f96f41e5', 'request_id': 'req-a7832f25-c002-48b8-b2fc-7fe814a563ba'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:48.902058 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05879-8720-4263-96f9-9430acc5f38b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:48.902729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05879-8720-4263-96f9-9430acc5f38b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:48.902729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05879-8720-4263-96f9-9430acc5f38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:48.927772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-07d05879-8720-4263-96f9-9430acc5f38b None None] MFA Rules not processed for user `677cbef82d164fa8bd0a24afc4012cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:48.929728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-07d05879-8720-4263-96f9-9430acc5f38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:48.933842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] The token cache is being invalidate because project 426c38bb690049b99a4d39c8f96f41e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:34:48.934436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:48.935243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a7832f25-c002-48b8-b2fc-7fe814a563ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:48.941660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1193/2427] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:34:48 2022] DELETE /identity/v3/projects/426c38bb690049b99a4d39c8f96f41e5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:48.947631 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad460d57-16c0-42e8-9bfd-62530e5274fe None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:48.947948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad460d57-16c0-42e8-9bfd-62530e5274fe None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:48.948229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad460d57-16c0-42e8-9bfd-62530e5274fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:48.978873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ad460d57-16c0-42e8-9bfd-62530e5274fe None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:48.980284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ad460d57-16c0-42e8-9bfd-62530e5274fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:49.036845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1235/2428] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:34:49.076906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1194/2429] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:49.085498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909111a8-d28f-4083-aa14-619f0c291ec1 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:49.085803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909111a8-d28f-4083-aa14-619f0c291ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:49.086093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909111a8-d28f-4083-aa14-619f0c291ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:49.125615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-909111a8-d28f-4083-aa14-619f0c291ec1 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:49.126963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-909111a8-d28f-4083-aa14-619f0c291ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:49.138044 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1236/2430] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:49.142685 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad460d57-16c0-42e8-9bfd-62530e5274fe None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:49.158141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad460d57-16c0-42e8-9bfd-62530e5274fe None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:49.163651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944faaf-644e-441d-a3ee-8bfa84f0c6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:49.164443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944faaf-644e-441d-a3ee-8bfa84f0c6bb admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:49.164824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944faaf-644e-441d-a3ee-8bfa84f0c6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:49.166207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944faaf-644e-441d-a3ee-8bfa84f0c6bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:49.166207 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944faaf-644e-441d-a3ee-8bfa84f0c6bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:34:49.166437 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944faaf-644e-441d-a3ee-8bfa84f0c6bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:49.168228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944faaf-644e-441d-a3ee-8bfa84f0c6bb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:49.188013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1195/2431] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:34:49.189140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909111a8-d28f-4083-aa14-619f0c291ec1 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:49.199150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909111a8-d28f-4083-aa14-619f0c291ec1 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:49.204829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac817267-ff91-42fe-8325-c1f74b447c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:49.208891 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac817267-ff91-42fe-8325-c1f74b447c5c admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:49.209188 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac817267-ff91-42fe-8325-c1f74b447c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:49.210285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac817267-ff91-42fe-8325-c1f74b447c5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:49.211417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac817267-ff91-42fe-8325-c1f74b447c5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:49.213804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac817267-ff91-42fe-8325-c1f74b447c5c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:49.360768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1237/2432] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:34:49 2022] POST /identity/v3/projects => generated 361 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:34:49.364670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944faaf-644e-441d-a3ee-8bfa84f0c6bb admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:49.381753 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944faaf-644e-441d-a3ee-8bfa84f0c6bb admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:49.460456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e5510-182b-4d2e-8b2a-90d03b1c5b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:49.461220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5510-182b-4d2e-8b2a-90d03b1c5b49 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:49.461633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5510-182b-4d2e-8b2a-90d03b1c5b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:49.462124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5510-182b-4d2e-8b2a-90d03b1c5b49 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:49.462781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e5510-182b-4d2e-8b2a-90d03b1c5b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:49.466223 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e5510-182b-4d2e-8b2a-90d03b1c5b49 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:49.562000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:49.570108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1196/2433] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:34:49 2022] POST /identity/v3/users => generated 281 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:49.575139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac817267-ff91-42fe-8325-c1f74b447c5c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:49.582871 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:49.588369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac817267-ff91-42fe-8325-c1f74b447c5c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:49.595905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13f731-d27a-457e-9e66-8513d45dd7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:49.596628 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13f731-d27a-457e-9e66-8513d45dd7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:49.596988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13f731-d27a-457e-9e66-8513d45dd7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:49.597367 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13f731-d27a-457e-9e66-8513d45dd7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:49.598228 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13f731-d27a-457e-9e66-8513d45dd7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:49.599918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13f731-d27a-457e-9e66-8513d45dd7a5 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:49.613361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1238/2434] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:49.616068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e5510-182b-4d2e-8b2a-90d03b1c5b49 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:49.635554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e5510-182b-4d2e-8b2a-90d03b1c5b49 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:49.641921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115e9e-d9d6-461c-8a8c-d310feeeea9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:49.643054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115e9e-d9d6-461c-8a8c-d310feeeea9f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:49.643557 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115e9e-d9d6-461c-8a8c-d310feeeea9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:49.644046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115e9e-d9d6-461c-8a8c-d310feeeea9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:49.645484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115e9e-d9d6-461c-8a8c-d310feeeea9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:49.647558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115e9e-d9d6-461c-8a8c-d310feeeea9f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:49.663403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1197/2435] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:49.667936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13f731-d27a-457e-9e66-8513d45dd7a5 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:49.682851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13f731-d27a-457e-9e66-8513d45dd7a5 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:49.687294 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b66f3-8024-42c6-aa70-5ea70a478b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:49.687941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b66f3-8024-42c6-aa70-5ea70a478b70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:49.688232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b66f3-8024-42c6-aa70-5ea70a478b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:49.688518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b66f3-8024-42c6-aa70-5ea70a478b70 admin admin] PATH_INFO: `/v3/projects/5744ef7b12b5479e8522a738f6273d73/users/9f8e3ddb7969413ba52741bb9d3e485d/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:49.705578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b66f3-8024-42c6-aa70-5ea70a478b70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5744ef7b12b5479e8522a738f6273d73, user_id=9f8e3ddb7969413ba52741bb9d3e485d, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:49.708549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b66f3-8024-42c6-aa70-5ea70a478b70 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:49.802936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1239/2436] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:49 2022] PUT /identity/v3/projects/5744ef7b12b5479e8522a738f6273d73/users/9f8e3ddb7969413ba52741bb9d3e485d/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:49.815643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fce6c8-e840-4ffe-97ed-f9a40f0f1f20 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:49.815950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fce6c8-e840-4ffe-97ed-f9a40f0f1f20 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:49.816234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fce6c8-e840-4ffe-97ed-f9a40f0f1f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:49.849070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-02fce6c8-e840-4ffe-97ed-f9a40f0f1f20 None None] MFA Rules not processed for user `9f8e3ddb7969413ba52741bb9d3e485d`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:49.850419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-02fce6c8-e840-4ffe-97ed-f9a40f0f1f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:49.948282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1198/2437] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:49.986436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b66f3-8024-42c6-aa70-5ea70a478b70 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:49.988526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-af5b66f3-8024-42c6-aa70-5ea70a478b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:50.072987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b66f3-8024-42c6-aa70-5ea70a478b70 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:50.077840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771893fb-aedc-4aa4-ab6e-b6bc0ec77181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:50.078546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771893fb-aedc-4aa4-ab6e-b6bc0ec77181 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:50.079151 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771893fb-aedc-4aa4-ab6e-b6bc0ec77181 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:50.079507 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771893fb-aedc-4aa4-ab6e-b6bc0ec77181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:50.093434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771893fb-aedc-4aa4-ab6e-b6bc0ec77181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:50.095677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771893fb-aedc-4aa4-ab6e-b6bc0ec77181 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:50.110896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1240/2438] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:50.129579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fce6c8-e840-4ffe-97ed-f9a40f0f1f20 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:50.219441 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fce6c8-e840-4ffe-97ed-f9a40f0f1f20 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:50.228482 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:50.229184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:50.231045 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:50.231356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] PATH_INFO: `/v3/users/9f8e3ddb7969413ba52741bb9d3e485d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:50.237589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f8e3ddb7969413ba52741bb9d3e485d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:50.239646 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:50.441830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f8e3ddb7969413ba52741bb9d3e485d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:50.457870 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f8e3ddb7969413ba52741bb9d3e485d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:50.466255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f8e3ddb7969413ba52741bb9d3e485d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:50.473197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:50.549866 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f8e3ddb7969413ba52741bb9d3e485d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:50.550611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f8e3ddb7969413ba52741bb9d3e485d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:50.561905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1199/2439] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:34:50 2022] DELETE /identity/v3/users/9f8e3ddb7969413ba52741bb9d3e485d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:50.566650 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764d87-2306-4e67-ae1b-e3f370558d55 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:50.566963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764d87-2306-4e67-ae1b-e3f370558d55 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:50.567214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764d87-2306-4e67-ae1b-e3f370558d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:50.598887 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e6764d87-2306-4e67-ae1b-e3f370558d55 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:50.600121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e6764d87-2306-4e67-ae1b-e3f370558d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:50.696464 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1241/2440] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:50.709081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:50.711062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:50.811911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55b2b2-25d3-4b33-b972-05f40cf79f68 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:50.815744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dbd28-ed75-472a-8d02-321c851c202b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:50.816440 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dbd28-ed75-472a-8d02-321c851c202b service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:50.816731 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dbd28-ed75-472a-8d02-321c851c202b service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:50.817016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dbd28-ed75-472a-8d02-321c851c202b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:50.840103 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dbd28-ed75-472a-8d02-321c851c202b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:50.845821 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dbd28-ed75-472a-8d02-321c851c202b service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:50.867172 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1200/2441] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:51.442978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6764d87-2306-4e67-ae1b-e3f370558d55 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:51.444592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e6764d87-2306-4e67-ae1b-e3f370558d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:51.463298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6764d87-2306-4e67-ae1b-e3f370558d55 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:51.467650 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:51.468554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:51.468986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:51.480161 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] PATH_INFO: `/v3/projects/5744ef7b12b5479e8522a738f6273d73` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:51.490426 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5744ef7b12b5479e8522a738f6273d73)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:51.492817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:51.621207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5744ef7b12b5479e8522a738f6273d73', 'request_id': 'req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:51.626877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5744ef7b12b5479e8522a738f6273d73', 'request_id': 'req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:51.938363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5744ef7b12b5479e8522a738f6273d73', 'request_id': 'req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:51.942336 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:51.953867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] The token cache is being invalidate because project 5744ef7b12b5479e8522a738f6273d73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:34:51.954167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:51.955222 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fcf1839-a1c8-4bcc-8f18-8aaa94d84a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:51.956805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1242/2442] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:34:51 2022] DELETE /identity/v3/projects/5744ef7b12b5479e8522a738f6273d73 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:51.962506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b4e77-59bd-4467-9214-c59df1783a4c None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:51.963140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b4e77-59bd-4467-9214-c59df1783a4c None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:51.963290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b4e77-59bd-4467-9214-c59df1783a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:51.985726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e79b4e77-59bd-4467-9214-c59df1783a4c None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:51.987059 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e79b4e77-59bd-4467-9214-c59df1783a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:52.051260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1201/2443] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:52.117606 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ac24f-975d-4371-861b-56df4b453797 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.117904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ac24f-975d-4371-861b-56df4b453797 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.118193 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ac24f-975d-4371-861b-56df4b453797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.141701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-d12ac24f-975d-4371-861b-56df4b453797 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:52.142940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d12ac24f-975d-4371-861b-56df4b453797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:52.153282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1243/2444] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:52.157837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b4e77-59bd-4467-9214-c59df1783a4c None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:52.168867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b4e77-59bd-4467-9214-c59df1783a4c None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:52.175123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de1839-08fd-4e32-a2b0-b7408991b8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:52.176000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1839-08fd-4e32-a2b0-b7408991b8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.176530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1839-08fd-4e32-a2b0-b7408991b8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.176889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1839-08fd-4e32-a2b0-b7408991b8a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.177719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1839-08fd-4e32-a2b0-b7408991b8a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:34:52.178123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1839-08fd-4e32-a2b0-b7408991b8a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:52.179814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1839-08fd-4e32-a2b0-b7408991b8a0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:52.193443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1202/2445] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:34:52.197811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ac24f-975d-4371-861b-56df4b453797 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:52.207479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ac24f-975d-4371-861b-56df4b453797 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:52.212433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad66f21-1ae8-4dab-b5a5-da8741caaaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:52.213048 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66f21-1ae8-4dab-b5a5-da8741caaaee admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.213365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66f21-1ae8-4dab-b5a5-da8741caaaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.213652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66f21-1ae8-4dab-b5a5-da8741caaaee admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.214317 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad66f21-1ae8-4dab-b5a5-da8741caaaee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:52.216232 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad66f21-1ae8-4dab-b5a5-da8741caaaee admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:52.329031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1244/2446] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:34:52 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:34:52.333397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de1839-08fd-4e32-a2b0-b7408991b8a0 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:52.344979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de1839-08fd-4e32-a2b0-b7408991b8a0 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:52.409336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668f6d4-21a1-4eea-940f-87fd5f28f14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:52.409336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668f6d4-21a1-4eea-940f-87fd5f28f14d admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.409336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668f6d4-21a1-4eea-940f-87fd5f28f14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.409336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668f6d4-21a1-4eea-940f-87fd5f28f14d admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.409861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668f6d4-21a1-4eea-940f-87fd5f28f14d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:52.409861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668f6d4-21a1-4eea-940f-87fd5f28f14d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:52.562570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1203/2447] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:34:52 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:34:52.563625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad66f21-1ae8-4dab-b5a5-da8741caaaee admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:52.578288 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad66f21-1ae8-4dab-b5a5-da8741caaaee admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:52.583233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef550ffc-98aa-4a0a-bb16-4f161e9587d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:52.583850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef550ffc-98aa-4a0a-bb16-4f161e9587d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.584132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef550ffc-98aa-4a0a-bb16-4f161e9587d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.584429 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef550ffc-98aa-4a0a-bb16-4f161e9587d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.585445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef550ffc-98aa-4a0a-bb16-4f161e9587d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:52.588078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef550ffc-98aa-4a0a-bb16-4f161e9587d3 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:52.606071 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1245/2448] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:34:52.610780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668f6d4-21a1-4eea-940f-87fd5f28f14d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:52.622080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668f6d4-21a1-4eea-940f-87fd5f28f14d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:52.627719 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad861095-395e-461a-9546-5c5be365b0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:52.628346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad861095-395e-461a-9546-5c5be365b0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.628633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad861095-395e-461a-9546-5c5be365b0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.628916 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad861095-395e-461a-9546-5c5be365b0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.629654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad861095-395e-461a-9546-5c5be365b0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:52.631274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad861095-395e-461a-9546-5c5be365b0fb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:52.645662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1204/2449] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:34:52.650255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef550ffc-98aa-4a0a-bb16-4f161e9587d3 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:52.663171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef550ffc-98aa-4a0a-bb16-4f161e9587d3 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:52.668446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84067f47-965f-44f2-bee8-397ce0a65b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:52.669168 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84067f47-965f-44f2-bee8-397ce0a65b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.669445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84067f47-965f-44f2-bee8-397ce0a65b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.669734 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84067f47-965f-44f2-bee8-397ce0a65b8a admin admin] PATH_INFO: `/v3/projects/e81f8c07f196451d91f4948c50f75173/users/ab35907f95894d37b943b4f6aca09dc0/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.691777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84067f47-965f-44f2-bee8-397ce0a65b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81f8c07f196451d91f4948c50f75173, user_id=ab35907f95894d37b943b4f6aca09dc0, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:52.694915 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84067f47-965f-44f2-bee8-397ce0a65b8a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:52.777358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1246/2450] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:34:52 2022] PUT /identity/v3/projects/e81f8c07f196451d91f4948c50f75173/users/ab35907f95894d37b943b4f6aca09dc0/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:52.787104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f7e5d-60bf-4bdd-ac4d-74cc0ab236ac None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.787743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f7e5d-60bf-4bdd-ac4d-74cc0ab236ac None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.788201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f7e5d-60bf-4bdd-ac4d-74cc0ab236ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.800267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:52.826022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-cb3f7e5d-60bf-4bdd-ac4d-74cc0ab236ac None None] MFA Rules not processed for user `ab35907f95894d37b943b4f6aca09dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:52.828120 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-cb3f7e5d-60bf-4bdd-ac4d-74cc0ab236ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:52.928088 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1205/2451] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:52.939515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84067f47-965f-44f2-bee8-397ce0a65b8a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:52.941246 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-84067f47-965f-44f2-bee8-397ce0a65b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:53.056906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84067f47-965f-44f2-bee8-397ce0a65b8a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:53.066866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b72fd9-c7fe-4562-948b-a44f19dd1223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:53.068370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72fd9-c7fe-4562-948b-a44f19dd1223 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:53.068370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72fd9-c7fe-4562-948b-a44f19dd1223 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:53.068370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72fd9-c7fe-4562-948b-a44f19dd1223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:53.078840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b72fd9-c7fe-4562-948b-a44f19dd1223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:53.080302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b72fd9-c7fe-4562-948b-a44f19dd1223 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:53.090297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1247/2452] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:53.771340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a676fc-2e92-4ab9-9e9c-b054fd09b8bd None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:53.771863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a676fc-2e92-4ab9-9e9c-b054fd09b8bd None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:53.771863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a676fc-2e92-4ab9-9e9c-b054fd09b8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:53.792063 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-e5a676fc-2e92-4ab9-9e9c-b054fd09b8bd None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:53.793282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e5a676fc-2e92-4ab9-9e9c-b054fd09b8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:53.869933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1206/2453] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:34:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:53.884589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b72fd9-c7fe-4562-948b-a44f19dd1223 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:53.886153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-75b72fd9-c7fe-4562-948b-a44f19dd1223 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:53.977427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b72fd9-c7fe-4562-948b-a44f19dd1223 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:53.984569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621f655-ad54-4366-81f7-5b822bff7c97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:53.985757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621f655-ad54-4366-81f7-5b822bff7c97 service swift] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:53.985872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621f655-ad54-4366-81f7-5b822bff7c97 service swift] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:53.986407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621f655-ad54-4366-81f7-5b822bff7c97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:53.997164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621f655-ad54-4366-81f7-5b822bff7c97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:53.999234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621f655-ad54-4366-81f7-5b822bff7c97 service swift] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:54.015417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1248/2454] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:34:57.464519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a676fc-2e92-4ab9-9e9c-b054fd09b8bd None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:57.557410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a676fc-2e92-4ab9-9e9c-b054fd09b8bd None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:57.563348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:57.564017 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:57.564303 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:57.564742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] PATH_INFO: `/v3/users/ab35907f95894d37b943b4f6aca09dc0` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:57.566087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab35907f95894d37b943b4f6aca09dc0)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:57.568253 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:58.227495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:58.232812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab35907f95894d37b943b4f6aca09dc0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:58.244799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab35907f95894d37b943b4f6aca09dc0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:58.253268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab35907f95894d37b943b4f6aca09dc0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:58.383892 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab35907f95894d37b943b4f6aca09dc0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:58.389722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab35907f95894d37b943b4f6aca09dc0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:58.398984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1207/2455] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:34:57 2022] DELETE /identity/v3/users/ab35907f95894d37b943b4f6aca09dc0 => generated 0 bytes in 935 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:34:58.403965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d8cec-fe4a-4b7b-aa5a-087ad44fb0ac None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:58.405287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d8cec-fe4a-4b7b-aa5a-087ad44fb0ac None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:58.405287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d8cec-fe4a-4b7b-aa5a-087ad44fb0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:58.426811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-108d8cec-fe4a-4b7b-aa5a-087ad44fb0ac None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:58.428375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-108d8cec-fe4a-4b7b-aa5a-087ad44fb0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:58.449966 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:58.520350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1249/2456] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:58.540679 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:58.542524 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:58.622272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0c88f-7e71-4d15-8f67-c1cbbd812277 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:58.627365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269df28-0301-44bd-acda-9cb073aaa952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:58.628006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269df28-0301-44bd-acda-9cb073aaa952 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:58.628293 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269df28-0301-44bd-acda-9cb073aaa952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:58.628574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269df28-0301-44bd-acda-9cb073aaa952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:58.640374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269df28-0301-44bd-acda-9cb073aaa952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:58.641926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269df28-0301-44bd-acda-9cb073aaa952 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:58.661777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1208/2457] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:59.283720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d8cec-fe4a-4b7b-aa5a-087ad44fb0ac None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:59.285364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-108d8cec-fe4a-4b7b-aa5a-087ad44fb0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:59.316127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d8cec-fe4a-4b7b-aa5a-087ad44fb0ac None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:59.323801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:59.326494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:59.326773 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:59.327175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] PATH_INFO: `/v3/projects/e81f8c07f196451d91f4948c50f75173` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:59.328460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81f8c07f196451d91f4948c50f75173)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:59.330482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:59.478042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81f8c07f196451d91f4948c50f75173', 'request_id': 'req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:59.486738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81f8c07f196451d91f4948c50f75173', 'request_id': 'req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:59.566379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81f8c07f196451d91f4948c50f75173', 'request_id': 'req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:59.580289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] The token cache is being invalidate because project e81f8c07f196451d91f4948c50f75173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:34:59.580631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:59.581364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ddef552d-fccc-4d96-9abe-b36fb2d25aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:34:59.582884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1250/2458] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:34:59 2022] DELETE /identity/v3/projects/e81f8c07f196451d91f4948c50f75173 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:35:08.654962 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=40,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:35:09.410992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86db86-2d18-445e-83ae-f3d57fcbd860 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:09.411294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86db86-2d18-445e-83ae-f3d57fcbd860 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:09.411582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86db86-2d18-445e-83ae-f3d57fcbd860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:09.435746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ff86db86-2d18-445e-83ae-f3d57fcbd860 None None] MFA Rules not processed for user `b1868c7d0ec8443b9ccb698ee02928b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:09.435746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ff86db86-2d18-445e-83ae-f3d57fcbd860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:09.514183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1209/2459] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:35:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:09.575836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:35:11.074903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f118-0dec-4c31-b41d-33e2de68c347 None None] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:11.075998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f118-0dec-4c31-b41d-33e2de68c347 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:11.077218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f118-0dec-4c31-b41d-33e2de68c347 None None] PATH_INFO: `/` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:11.079318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1251/2460] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 28 06:35:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:35:11.088323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c638f-019c-43d9-8e20-7fc3e946b64d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:11.088323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c638f-019c-43d9-8e20-7fc3e946b64d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:11.088921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c638f-019c-43d9-8e20-7fc3e946b64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:11.114076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b30c638f-019c-43d9-8e20-7fc3e946b64d None None] MFA Rules not processed for user `5706aa2d6bf54f05badf9588f51f4677`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:11.115335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b30c638f-019c-43d9-8e20-7fc3e946b64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:11.217944 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1210/2461] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 28 06:35:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:11.231423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663f118-0dec-4c31-b41d-33e2de68c347 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:11.232851 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9663f118-0dec-4c31-b41d-33e2de68c347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:11.327402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663f118-0dec-4c31-b41d-33e2de68c347 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:11.332654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b7330-36a2-4fac-8bbc-b584e2c81140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:11.333299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b7330-36a2-4fac-8bbc-b584e2c81140 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:11.333633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b7330-36a2-4fac-8bbc-b584e2c81140 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:11.333900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b7330-36a2-4fac-8bbc-b584e2c81140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:11.347937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b7330-36a2-4fac-8bbc-b584e2c81140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:11.349691 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b7330-36a2-4fac-8bbc-b584e2c81140 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:11.364262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1252/2462] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:15.795728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9169d-2511-4b45-8704-a38cb23359f7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:15.796347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9169d-2511-4b45-8704-a38cb23359f7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:15.796617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9169d-2511-4b45-8704-a38cb23359f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:15.817976 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c2f9169d-2511-4b45-8704-a38cb23359f7 None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:15.819269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c2f9169d-2511-4b45-8704-a38cb23359f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:15.911942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1211/2463] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 28 06:35:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:15.912155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b7330-36a2-4fac-8bbc-b584e2c81140 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:15.912922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8c6b7330-36a2-4fac-8bbc-b584e2c81140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:16.004443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b7330-36a2-4fac-8bbc-b584e2c81140 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:16.009770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb36671-627c-4a5c-8ef6-22bd1dd20642 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:16.010404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36671-627c-4a5c-8ef6-22bd1dd20642 service swift] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:16.010693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36671-627c-4a5c-8ef6-22bd1dd20642 service swift] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:16.010973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36671-627c-4a5c-8ef6-22bd1dd20642 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:16.020642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb36671-627c-4a5c-8ef6-22bd1dd20642 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:16.022292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb36671-627c-4a5c-8ef6-22bd1dd20642 service swift] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:16.031942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1253/2464] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:35:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:35:25.890464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=81 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:35:26.028713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=58 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:35:30.786814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02118246-c155-4248-9145-e05e4a1d4c3d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:30.787302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02118246-c155-4248-9145-e05e4a1d4c3d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:30.787515 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02118246-c155-4248-9145-e05e4a1d4c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:30.808700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-02118246-c155-4248-9145-e05e4a1d4c3d None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:30.809963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-02118246-c155-4248-9145-e05e4a1d4c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:30.817889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1212/2465] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 28 06:35:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:30.830057 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb36671-627c-4a5c-8ef6-22bd1dd20642 service swift] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:30.840919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb36671-627c-4a5c-8ef6-22bd1dd20642 service swift] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:30.846103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb1f2c-d642-4092-97af-80f6d0866fc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:30.846704 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb1f2c-d642-4092-97af-80f6d0866fc3 service swift] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:30.846988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb1f2c-d642-4092-97af-80f6d0866fc3 service swift] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:30.847272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb1f2c-d642-4092-97af-80f6d0866fc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:30.855293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb1f2c-d642-4092-97af-80f6d0866fc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:30.856869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb1f2c-d642-4092-97af-80f6d0866fc3 service swift] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:30.865453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1254/2466] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:35:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:35:40.802255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:35:40.861659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=6 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:35:43.445508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02118246-c155-4248-9145-e05e4a1d4c3d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:43.446964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-02118246-c155-4248-9145-e05e4a1d4c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:43.459083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02118246-c155-4248-9145-e05e4a1d4c3d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:43.462644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961a6b9-0563-467d-9b5e-7f197828d098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:43.463257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961a6b9-0563-467d-9b5e-7f197828d098 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:43.463540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961a6b9-0563-467d-9b5e-7f197828d098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:43.463815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961a6b9-0563-467d-9b5e-7f197828d098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:43.465605 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6961a6b9-0563-467d-9b5e-7f197828d098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:43.532193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961a6b9-0563-467d-9b5e-7f197828d098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:43.533583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961a6b9-0563-467d-9b5e-7f197828d098 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:43.544074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1213/2467] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:35:53.539742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=30 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:35:54.913635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb1f2c-d642-4092-97af-80f6d0866fc3 service swift] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:54.926565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb1f2c-d642-4092-97af-80f6d0866fc3 service swift] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:54.930794 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97a36c-373c-4f4c-9277-4a18304ae3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:54.931419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97a36c-373c-4f4c-9277-4a18304ae3dc service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:54.931705 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97a36c-373c-4f4c-9277-4a18304ae3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:54.932005 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97a36c-373c-4f4c-9277-4a18304ae3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:54.933850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3c97a36c-373c-4f4c-9277-4a18304ae3dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:55.004633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97a36c-373c-4f4c-9277-4a18304ae3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:55.006119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97a36c-373c-4f4c-9277-4a18304ae3dc service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:55.017004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1255/2468] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:58.908076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961a6b9-0563-467d-9b5e-7f197828d098 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:58.910491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6961a6b9-0563-467d-9b5e-7f197828d098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:58.990745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961a6b9-0563-467d-9b5e-7f197828d098 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:58.995891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbbe03-0ae9-41b6-afeb-4e7b204691ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:58.996512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbbe03-0ae9-41b6-afeb-4e7b204691ea service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:58.996803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbbe03-0ae9-41b6-afeb-4e7b204691ea service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:58.997084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbbe03-0ae9-41b6-afeb-4e7b204691ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:58.998935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e4fbbe03-0ae9-41b6-afeb-4e7b204691ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:59.010680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbbe03-0ae9-41b6-afeb-4e7b204691ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:59.012383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbbe03-0ae9-41b6-afeb-4e7b204691ea service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:59.023785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1214/2469] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:59.124898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97a36c-373c-4f4c-9277-4a18304ae3dc service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:59.126956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3c97a36c-373c-4f4c-9277-4a18304ae3dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:59.141391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97a36c-373c-4f4c-9277-4a18304ae3dc service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:59.147767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:59.148450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:59.149708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:59.149708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] PATH_INFO: `/v3/users/61ad8186d91848bc8be42e3be51e9a75` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:59.150631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=61ad8186d91848bc8be42e3be51e9a75)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:59.152880 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:59.397866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61ad8186d91848bc8be42e3be51e9a75'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:35:59.410768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61ad8186d91848bc8be42e3be51e9a75'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:35:59.411471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61ad8186d91848bc8be42e3be51e9a75'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:35:59.418463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61ad8186d91848bc8be42e3be51e9a75'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:35:59.479744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61ad8186d91848bc8be42e3be51e9a75'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:35:59.488908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1256/2470] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:35:59 2022] DELETE /identity/v3/users/61ad8186d91848bc8be42e3be51e9a75 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:35:59.555413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbbe03-0ae9-41b6-afeb-4e7b204691ea service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:59.566105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbbe03-0ae9-41b6-afeb-4e7b204691ea service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:59.624006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e917634-8256-44eb-876c-af7ba9892d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:59.624637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e917634-8256-44eb-876c-af7ba9892d93 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:59.624924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e917634-8256-44eb-876c-af7ba9892d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:59.625204 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e917634-8256-44eb-876c-af7ba9892d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:59.627005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2e917634-8256-44eb-876c-af7ba9892d93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:59.692560 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e917634-8256-44eb-876c-af7ba9892d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:59.693964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e917634-8256-44eb-876c-af7ba9892d93 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:59.703828 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1215/2471] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:59.782615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:59.784510 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:59.802292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1c0d9-f26a-478f-9794-4ed3d4eb3d7b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:59.809476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:59.810094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:59.810386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:59.810667 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] PATH_INFO: `/v3/projects/85bf0de611024c06aee30418f26d2f1c` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:59.811900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85bf0de611024c06aee30418f26d2f1c)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:59.814078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:59.927387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85bf0de611024c06aee30418f26d2f1c', 'request_id': 'req-bc5ed83e-b79b-4d6a-b435-1660ec959355'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:35:59.933220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85bf0de611024c06aee30418f26d2f1c', 'request_id': 'req-bc5ed83e-b79b-4d6a-b435-1660ec959355'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:35:59.936617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:00.034785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85bf0de611024c06aee30418f26d2f1c', 'request_id': 'req-bc5ed83e-b79b-4d6a-b435-1660ec959355'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:00.047379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] The token cache is being invalidate because project 85bf0de611024c06aee30418f26d2f1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:00.047754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:00.048426 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-bc5ed83e-b79b-4d6a-b435-1660ec959355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:00.050007 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1257/2472] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:35:59 2022] DELETE /identity/v3/projects/85bf0de611024c06aee30418f26d2f1c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:00.055144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e80a1f-862e-4cff-b423-ae155b8258b6 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:00.055681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e80a1f-862e-4cff-b423-ae155b8258b6 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:00.055681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e80a1f-862e-4cff-b423-ae155b8258b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:00.075736 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f1e80a1f-862e-4cff-b423-ae155b8258b6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:00.076988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f1e80a1f-862e-4cff-b423-ae155b8258b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:00.113733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:00.141076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1216/2473] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:00.149581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75de1d2-15d3-4504-bb75-9e4b39d0862e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:00.150165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75de1d2-15d3-4504-bb75-9e4b39d0862e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:00.150165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75de1d2-15d3-4504-bb75-9e4b39d0862e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:00.170885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-a75de1d2-15d3-4504-bb75-9e4b39d0862e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:00.172116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a75de1d2-15d3-4504-bb75-9e4b39d0862e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:00.181084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1258/2474] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:00.185576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e80a1f-862e-4cff-b423-ae155b8258b6 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:00.195746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e80a1f-862e-4cff-b423-ae155b8258b6 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:00.200427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879b6f-3faa-4470-ac9d-2cb7166edce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:00.201048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879b6f-3faa-4470-ac9d-2cb7166edce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:00.201517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879b6f-3faa-4470-ac9d-2cb7166edce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:00.201781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879b6f-3faa-4470-ac9d-2cb7166edce7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:00.202486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6879b6f-3faa-4470-ac9d-2cb7166edce7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:00.202793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6879b6f-3faa-4470-ac9d-2cb7166edce7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:00.204421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6879b6f-3faa-4470-ac9d-2cb7166edce7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:00.220630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1217/2475] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:36:00.222625 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75de1d2-15d3-4504-bb75-9e4b39d0862e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:00.232516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75de1d2-15d3-4504-bb75-9e4b39d0862e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:00.238285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13fb51-cdda-4323-9ecc-83cba0845bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:00.238997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13fb51-cdda-4323-9ecc-83cba0845bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:00.239284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13fb51-cdda-4323-9ecc-83cba0845bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:00.239564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13fb51-cdda-4323-9ecc-83cba0845bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:00.240254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13fb51-cdda-4323-9ecc-83cba0845bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:00.242247 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13fb51-cdda-4323-9ecc-83cba0845bb0 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:00.335783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1259/2476] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:00 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:00.340061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879b6f-3faa-4470-ac9d-2cb7166edce7 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:00.357588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879b6f-3faa-4470-ac9d-2cb7166edce7 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:00.424094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61af2a1-87d2-40b2-a60a-000e072dfbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:00.425564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61af2a1-87d2-40b2-a60a-000e072dfbd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:00.426042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61af2a1-87d2-40b2-a60a-000e072dfbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:00.426328 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61af2a1-87d2-40b2-a60a-000e072dfbd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:00.427029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61af2a1-87d2-40b2-a60a-000e072dfbd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:00.429034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61af2a1-87d2-40b2-a60a-000e072dfbd3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:00.509645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1218/2477] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:00 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:00.514080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13fb51-cdda-4323-9ecc-83cba0845bb0 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:00.525739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13fb51-cdda-4323-9ecc-83cba0845bb0 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:00.531020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7987b7f-688b-4476-a5f4-367289e045c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:00.531661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7987b7f-688b-4476-a5f4-367289e045c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:00.532097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7987b7f-688b-4476-a5f4-367289e045c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:00.532388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7987b7f-688b-4476-a5f4-367289e045c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:00.533109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7987b7f-688b-4476-a5f4-367289e045c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:00.535477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7987b7f-688b-4476-a5f4-367289e045c4 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:00.546542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1260/2478] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:00.551436 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61af2a1-87d2-40b2-a60a-000e072dfbd3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:00.562481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61af2a1-87d2-40b2-a60a-000e072dfbd3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:00.568257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435539a6-d2bf-44d5-97c8-4d0436905e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:00.568910 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435539a6-d2bf-44d5-97c8-4d0436905e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:00.569204 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435539a6-d2bf-44d5-97c8-4d0436905e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:00.569527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435539a6-d2bf-44d5-97c8-4d0436905e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:00.570245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435539a6-d2bf-44d5-97c8-4d0436905e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:00.572044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435539a6-d2bf-44d5-97c8-4d0436905e4c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:00.587425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1219/2479] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:00.588914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7987b7f-688b-4476-a5f4-367289e045c4 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:00.599983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7987b7f-688b-4476-a5f4-367289e045c4 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:00.604900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ca78f-3cff-4d32-a961-081dfa931be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:00.605635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca78f-3cff-4d32-a961-081dfa931be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:00.605928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca78f-3cff-4d32-a961-081dfa931be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:00.606218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca78f-3cff-4d32-a961-081dfa931be9 admin admin] PATH_INFO: `/v3/projects/d6b119ed685b4285b2c7ef69e3df53c4/users/c5e013bf52fa45dc8e4b65f183506cf7/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:00.624165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ca78f-3cff-4d32-a961-081dfa931be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b119ed685b4285b2c7ef69e3df53c4, user_id=c5e013bf52fa45dc8e4b65f183506cf7, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:00.629573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ca78f-3cff-4d32-a961-081dfa931be9 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:00.749002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1261/2480] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:00 2022] PUT /identity/v3/projects/d6b119ed685b4285b2c7ef69e3df53c4/users/c5e013bf52fa45dc8e4b65f183506cf7/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:00.761574 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575840-fc51-4811-9380-c75573d8ffe3 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:00.761928 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575840-fc51-4811-9380-c75573d8ffe3 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:00.762188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575840-fc51-4811-9380-c75573d8ffe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:00.803553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0b575840-fc51-4811-9380-c75573d8ffe3 None None] MFA Rules not processed for user `c5e013bf52fa45dc8e4b65f183506cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:00.803553 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0b575840-fc51-4811-9380-c75573d8ffe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:00.867199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1220/2481] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:00.879542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ca78f-3cff-4d32-a961-081dfa931be9 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:00.881293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-812ca78f-3cff-4d32-a961-081dfa931be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:00.956223 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ca78f-3cff-4d32-a961-081dfa931be9 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:00.962796 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76af28-0b78-4566-a46b-0ae8add25607 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:00.963447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76af28-0b78-4566-a46b-0ae8add25607 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:00.963736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76af28-0b78-4566-a46b-0ae8add25607 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:00.964022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76af28-0b78-4566-a46b-0ae8add25607 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:00.972488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76af28-0b78-4566-a46b-0ae8add25607 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:00.974133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76af28-0b78-4566-a46b-0ae8add25607 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:00.983414 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1262/2482] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:03.713038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b575840-fc51-4811-9380-c75573d8ffe3 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:03.714517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0b575840-fc51-4811-9380-c75573d8ffe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:03.757106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:03.788115 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b575840-fc51-4811-9380-c75573d8ffe3 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:03.793150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0cac0-d5d9-484a-881f-47931bad3d0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:03.793806 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0cac0-d5d9-484a-881f-47931bad3d0c service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:03.794094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0cac0-d5d9-484a-881f-47931bad3d0c service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:03.794375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0cac0-d5d9-484a-881f-47931bad3d0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:03.796162 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4cb0cac0-d5d9-484a-881f-47931bad3d0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:03.808112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0cac0-d5d9-484a-881f-47931bad3d0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:03.809764 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0cac0-d5d9-484a-881f-47931bad3d0c service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:03.820374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1221/2483] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:03.838389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76af28-0b78-4566-a46b-0ae8add25607 service nova] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:03.838389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-df76af28-0b78-4566-a46b-0ae8add25607 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:03.914329 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76af28-0b78-4566-a46b-0ae8add25607 service nova] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:03.918982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:03.919629 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:03.919917 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:03.920233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] PATH_INFO: `/v3/users/0d268e37c8b64bcc9261a80bb0461edc` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:03.921441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d268e37c8b64bcc9261a80bb0461edc)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:03.923381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:03.936268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:04.140139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d268e37c8b64bcc9261a80bb0461edc'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:04.152877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d268e37c8b64bcc9261a80bb0461edc'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:04.153638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d268e37c8b64bcc9261a80bb0461edc'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:04.161166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d268e37c8b64bcc9261a80bb0461edc'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:04.262573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d0cac64a-3800-4890-9639-fe5109d22a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d268e37c8b64bcc9261a80bb0461edc'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:04.270351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1263/2484] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:03 2022] DELETE /identity/v3/users/0d268e37c8b64bcc9261a80bb0461edc => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:04.587385 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0cac0-d5d9-484a-881f-47931bad3d0c service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:04.599689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0cac0-d5d9-484a-881f-47931bad3d0c service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:04.660304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:04.660950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:04.661235 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:04.661540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] PATH_INFO: `/v3/projects/f7bae530a50e4c6ea97a654750ddf79f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:04.662701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7bae530a50e4c6ea97a654750ddf79f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:04.664616 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:04.766957 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7bae530a50e4c6ea97a654750ddf79f', 'request_id': 'req-1aa00925-55e7-4582-9aad-c3af53ccfaba'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:04.773071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7bae530a50e4c6ea97a654750ddf79f', 'request_id': 'req-1aa00925-55e7-4582-9aad-c3af53ccfaba'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:04.779264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7bae530a50e4c6ea97a654750ddf79f', 'request_id': 'req-1aa00925-55e7-4582-9aad-c3af53ccfaba'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:04.850773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] The token cache is being invalidate because project f7bae530a50e4c6ea97a654750ddf79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:04.851060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:04.851681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1aa00925-55e7-4582-9aad-c3af53ccfaba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:04.853194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1222/2485] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:04 2022] DELETE /identity/v3/projects/f7bae530a50e4c6ea97a654750ddf79f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:04.858127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256618ab-7f83-4d11-8283-2a5246dc1339 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:04.858641 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256618ab-7f83-4d11-8283-2a5246dc1339 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:04.858742 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256618ab-7f83-4d11-8283-2a5246dc1339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:04.879076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-256618ab-7f83-4d11-8283-2a5246dc1339 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:04.880303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-256618ab-7f83-4d11-8283-2a5246dc1339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:04.941109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1264/2486] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:04.949187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87859b2f-7c64-44f9-b894-9fc0a8d34cde None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:04.949519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87859b2f-7c64-44f9-b894-9fc0a8d34cde None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:04.949803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87859b2f-7c64-44f9-b894-9fc0a8d34cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:04.970872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-87859b2f-7c64-44f9-b894-9fc0a8d34cde None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:04.972266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-87859b2f-7c64-44f9-b894-9fc0a8d34cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:04.980603 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1223/2487] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:04.984868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256618ab-7f83-4d11-8283-2a5246dc1339 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:04.995287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256618ab-7f83-4d11-8283-2a5246dc1339 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:05.000191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514be4f3-aeb7-42e8-b454-23a24ee386f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:05.000821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514be4f3-aeb7-42e8-b454-23a24ee386f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:05.001106 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514be4f3-aeb7-42e8-b454-23a24ee386f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:05.001417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514be4f3-aeb7-42e8-b454-23a24ee386f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:05.002132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514be4f3-aeb7-42e8-b454-23a24ee386f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:05.003080 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514be4f3-aeb7-42e8-b454-23a24ee386f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:05.004700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514be4f3-aeb7-42e8-b454-23a24ee386f8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:05.016801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1265/2488] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:36:05.022717 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87859b2f-7c64-44f9-b894-9fc0a8d34cde None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:05.032397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87859b2f-7c64-44f9-b894-9fc0a8d34cde None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:05.037209 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516503d-de27-4f7f-8033-a94a7972b096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:05.037863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516503d-de27-4f7f-8033-a94a7972b096 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:05.038151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516503d-de27-4f7f-8033-a94a7972b096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:05.038434 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516503d-de27-4f7f-8033-a94a7972b096 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:05.039135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516503d-de27-4f7f-8033-a94a7972b096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:05.041068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516503d-de27-4f7f-8033-a94a7972b096 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:05.132604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1224/2489] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:05 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:05.136763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514be4f3-aeb7-42e8-b454-23a24ee386f8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:05.147622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514be4f3-aeb7-42e8-b454-23a24ee386f8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:05.205263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fc09a-b3f8-4b1f-b97e-f9ba8fd64d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:05.206393 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fc09a-b3f8-4b1f-b97e-f9ba8fd64d31 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:05.206680 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fc09a-b3f8-4b1f-b97e-f9ba8fd64d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:05.206968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fc09a-b3f8-4b1f-b97e-f9ba8fd64d31 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:05.207631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fc09a-b3f8-4b1f-b97e-f9ba8fd64d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:05.209855 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fc09a-b3f8-4b1f-b97e-f9ba8fd64d31 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:05.314448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1266/2490] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:05 2022] POST /identity/v3/users => generated 282 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:05.319807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516503d-de27-4f7f-8033-a94a7972b096 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:05.331599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516503d-de27-4f7f-8033-a94a7972b096 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:05.336909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a997b36-35bd-4ebe-b277-cc5029079395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:05.337833 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a997b36-35bd-4ebe-b277-cc5029079395 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:05.338118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a997b36-35bd-4ebe-b277-cc5029079395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:05.338393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a997b36-35bd-4ebe-b277-cc5029079395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:05.339096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a997b36-35bd-4ebe-b277-cc5029079395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:05.340765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a997b36-35bd-4ebe-b277-cc5029079395 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:05.352575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1225/2491] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:05.362871 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fc09a-b3f8-4b1f-b97e-f9ba8fd64d31 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:05.374469 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fc09a-b3f8-4b1f-b97e-f9ba8fd64d31 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:05.380004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8ca7a-239b-449d-a9f5-0f6fb6a18b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:05.380659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8ca7a-239b-449d-a9f5-0f6fb6a18b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:05.380949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8ca7a-239b-449d-a9f5-0f6fb6a18b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:05.381252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8ca7a-239b-449d-a9f5-0f6fb6a18b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:05.381988 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8ca7a-239b-449d-a9f5-0f6fb6a18b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:05.383596 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8ca7a-239b-449d-a9f5-0f6fb6a18b19 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:05.385053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:05.396551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1267/2492] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:05.401911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a997b36-35bd-4ebe-b277-cc5029079395 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:05.413025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a997b36-35bd-4ebe-b277-cc5029079395 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:05.417928 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea710e-b4ee-481b-bdbd-75c1187840e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:05.418585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea710e-b4ee-481b-bdbd-75c1187840e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:05.418874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea710e-b4ee-481b-bdbd-75c1187840e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:05.419162 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea710e-b4ee-481b-bdbd-75c1187840e8 admin admin] PATH_INFO: `/v3/projects/dff4a79b277e425ea9d1e651aff326a0/users/8c1973d358084896bb3b8208747aca6d/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:05.432580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:05.435656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea710e-b4ee-481b-bdbd-75c1187840e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff4a79b277e425ea9d1e651aff326a0, user_id=8c1973d358084896bb3b8208747aca6d, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:05.438451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea710e-b4ee-481b-bdbd-75c1187840e8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:05.528466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1226/2493] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:05 2022] PUT /identity/v3/projects/dff4a79b277e425ea9d1e651aff326a0/users/8c1973d358084896bb3b8208747aca6d/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:05.542875 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ddf4c-4bc8-4a59-a7c8-1fab3d53c6af None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:05.543170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ddf4c-4bc8-4a59-a7c8-1fab3d53c6af None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:05.543460 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ddf4c-4bc8-4a59-a7c8-1fab3d53c6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:05.581192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-764ddf4c-4bc8-4a59-a7c8-1fab3d53c6af None None] MFA Rules not processed for user `8c1973d358084896bb3b8208747aca6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:05.583729 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-764ddf4c-4bc8-4a59-a7c8-1fab3d53c6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:05.661526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1268/2494] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:05.666135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea710e-b4ee-481b-bdbd-75c1187840e8 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:05.731318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea710e-b4ee-481b-bdbd-75c1187840e8 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:05.736029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0852d-1ae5-4a95-a0ca-58d85943f194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:05.736671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0852d-1ae5-4a95-a0ca-58d85943f194 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:05.736955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0852d-1ae5-4a95-a0ca-58d85943f194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:05.737227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0852d-1ae5-4a95-a0ca-58d85943f194 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:05.737932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0852d-1ae5-4a95-a0ca-58d85943f194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:05.739965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0852d-1ae5-4a95-a0ca-58d85943f194 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:05.847179 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1227/2495] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:05 2022] POST /identity/v3/projects => generated 363 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:05.851405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ddf4c-4bc8-4a59-a7c8-1fab3d53c6af None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:05.863316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ddf4c-4bc8-4a59-a7c8-1fab3d53c6af None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:05.920363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0372ef-a811-42a9-b336-23347810dca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:05.920990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0372ef-a811-42a9-b336-23347810dca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:05.921279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0372ef-a811-42a9-b336-23347810dca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:05.921655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0372ef-a811-42a9-b336-23347810dca9 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:05.922338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0372ef-a811-42a9-b336-23347810dca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:05.925017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0372ef-a811-42a9-b336-23347810dca9 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:05.998889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1269/2496] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:05 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:06.003042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0852d-1ae5-4a95-a0ca-58d85943f194 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:06.013403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0852d-1ae5-4a95-a0ca-58d85943f194 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:06.018624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb1556-4922-4bc6-8e4a-c325ce269ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:06.019236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb1556-4922-4bc6-8e4a-c325ce269ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:06.019545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb1556-4922-4bc6-8e4a-c325ce269ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:06.019839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb1556-4922-4bc6-8e4a-c325ce269ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:06.020545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb1556-4922-4bc6-8e4a-c325ce269ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:06.022218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb1556-4922-4bc6-8e4a-c325ce269ddb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:06.033147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1228/2497] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:06.038184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0372ef-a811-42a9-b336-23347810dca9 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:06.048445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0372ef-a811-42a9-b336-23347810dca9 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:06.054516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb8d3b-9e5b-47de-b034-6d79ef5ed8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:06.056090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb8d3b-9e5b-47de-b034-6d79ef5ed8b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:06.056090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb8d3b-9e5b-47de-b034-6d79ef5ed8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:06.056090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb8d3b-9e5b-47de-b034-6d79ef5ed8b8 admin admin] PATH_INFO: `/v3/projects/df50f68455d54145a6e395f74b699068/users/73a75c10f68c4600aca56bccc4d62221/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:06.071533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb8d3b-9e5b-47de-b034-6d79ef5ed8b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df50f68455d54145a6e395f74b699068, user_id=73a75c10f68c4600aca56bccc4d62221, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:06.074246 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb8d3b-9e5b-47de-b034-6d79ef5ed8b8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:06.159274 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1270/2498] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:06 2022] PUT /identity/v3/projects/df50f68455d54145a6e395f74b699068/users/73a75c10f68c4600aca56bccc4d62221/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:06.163734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb1556-4922-4bc6-8e4a-c325ce269ddb admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:06.174533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb1556-4922-4bc6-8e4a-c325ce269ddb admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:06.232353 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2fee5-7db5-4f3c-b2ea-9feb090e5a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:06.232974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2fee5-7db5-4f3c-b2ea-9feb090e5a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:06.233260 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2fee5-7db5-4f3c-b2ea-9feb090e5a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:06.233572 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2fee5-7db5-4f3c-b2ea-9feb090e5a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:06.234267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2fee5-7db5-4f3c-b2ea-9feb090e5a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:06.235893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2fee5-7db5-4f3c-b2ea-9feb090e5a53 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:06.245942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1229/2499] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:06.250418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb8d3b-9e5b-47de-b034-6d79ef5ed8b8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:06.261697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb8d3b-9e5b-47de-b034-6d79ef5ed8b8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:06.266400 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3a903-c35d-44d3-900a-67b618aa789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:06.267086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3a903-c35d-44d3-900a-67b618aa789e admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:06.267370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3a903-c35d-44d3-900a-67b618aa789e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:06.267647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3a903-c35d-44d3-900a-67b618aa789e admin admin] PATH_INFO: `/v3/projects/df50f68455d54145a6e395f74b699068/users/73a75c10f68c4600aca56bccc4d62221/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:06.270231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3a903-c35d-44d3-900a-67b618aa789e admin admin] RBAC: Authorizing `identity:create_grant(project_id=df50f68455d54145a6e395f74b699068, user_id=73a75c10f68c4600aca56bccc4d62221, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:06.272918 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3a903-c35d-44d3-900a-67b618aa789e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:06.297603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1271/2500] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:06 2022] PUT /identity/v3/projects/df50f68455d54145a6e395f74b699068/users/73a75c10f68c4600aca56bccc4d62221/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:06.308384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f2407-4c61-4713-9c61-f2458eed09a7 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:06.308678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f2407-4c61-4713-9c61-f2458eed09a7 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:06.308964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f2407-4c61-4713-9c61-f2458eed09a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:06.348012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fb2f2407-4c61-4713-9c61-f2458eed09a7 None None] MFA Rules not processed for user `73a75c10f68c4600aca56bccc4d62221`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:06.349236 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fb2f2407-4c61-4713-9c61-f2458eed09a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:06.397593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:06.450876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1230/2501] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:06 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:06.460567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3a903-c35d-44d3-900a-67b618aa789e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:06.462493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-86a3a903-c35d-44d3-900a-67b618aa789e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:06.495104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:06.525073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3a903-c35d-44d3-900a-67b618aa789e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:06.528808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2b2be-505b-452a-a627-7aa19778bdab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:06.529594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2b2be-505b-452a-a627-7aa19778bdab service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:06.529878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2b2be-505b-452a-a627-7aa19778bdab service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:06.530159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2b2be-505b-452a-a627-7aa19778bdab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:06.540572 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2b2be-505b-452a-a627-7aa19778bdab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:06.542056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2b2be-505b-452a-a627-7aa19778bdab service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:06.552744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1272/2502] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:06 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:07.531182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f2407-4c61-4713-9c61-f2458eed09a7 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:07.616696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f2407-4c61-4713-9c61-f2458eed09a7 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:07.624843 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:07.625533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:07.625824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:07.631757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] PATH_INFO: `/v3/users/8c1973d358084896bb3b8208747aca6d` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:07.631757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c1973d358084896bb3b8208747aca6d)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:07.631757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:07.813226 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c1973d358084896bb3b8208747aca6d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:07.831481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c1973d358084896bb3b8208747aca6d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:07.839190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c1973d358084896bb3b8208747aca6d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:07.953891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c1973d358084896bb3b8208747aca6d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:07.954634 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c1973d358084896bb3b8208747aca6d'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:07.961622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1231/2503] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:07 2022] DELETE /identity/v3/users/8c1973d358084896bb3b8208747aca6d => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:07.967078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02d36c-05dc-468c-abbe-7a48d01e8dde None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:07.967078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02d36c-05dc-468c-abbe-7a48d01e8dde None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:07.967078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02d36c-05dc-468c-abbe-7a48d01e8dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:07.998196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-6d02d36c-05dc-468c-abbe-7a48d01e8dde None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:07.999533 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6d02d36c-05dc-468c-abbe-7a48d01e8dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:08.066944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1273/2504] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:08.076521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:08.078087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:08.102369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:08.161251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffafbe3-eb42-49f2-a46a-e5be05372943 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:08.165050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d1536-f40e-415e-8e95-8390585025f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:08.165703 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d1536-f40e-415e-8e95-8390585025f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:08.165993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d1536-f40e-415e-8e95-8390585025f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:08.166274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d1536-f40e-415e-8e95-8390585025f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:08.175600 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d1536-f40e-415e-8e95-8390585025f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:08.177042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d1536-f40e-415e-8e95-8390585025f8 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:08.188246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1232/2505] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:08.639591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02d36c-05dc-468c-abbe-7a48d01e8dde None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:08.641238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6d02d36c-05dc-468c-abbe-7a48d01e8dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:08.655924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02d36c-05dc-468c-abbe-7a48d01e8dde None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:08.661084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:08.661920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:08.662209 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:08.662494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] PATH_INFO: `/v3/projects/dff4a79b277e425ea9d1e651aff326a0` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:08.663705 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff4a79b277e425ea9d1e651aff326a0)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:08.666016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:08.788835 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff4a79b277e425ea9d1e651aff326a0', 'request_id': 'req-60af7322-6d6d-4818-8619-c806795e80d5'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:08.794672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff4a79b277e425ea9d1e651aff326a0', 'request_id': 'req-60af7322-6d6d-4818-8619-c806795e80d5'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:08.874326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff4a79b277e425ea9d1e651aff326a0', 'request_id': 'req-60af7322-6d6d-4818-8619-c806795e80d5'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:08.888577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] The token cache is being invalidate because project dff4a79b277e425ea9d1e651aff326a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:08.888880 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:08.889578 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:08.891677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1274/2506] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:08 2022] DELETE /identity/v3/projects/dff4a79b277e425ea9d1e651aff326a0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:08.897018 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d1536-f40e-415e-8e95-8390585025f8 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:08.907470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d1536-f40e-415e-8e95-8390585025f8 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:08.972949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:08.973649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:08.973961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:08.974279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] PATH_INFO: `/v3/users/73a75c10f68c4600aca56bccc4d62221` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:08.975434 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a75c10f68c4600aca56bccc4d62221)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:08.977461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:09.133079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a75c10f68c4600aca56bccc4d62221'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:09.152775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a75c10f68c4600aca56bccc4d62221'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:09.163389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a75c10f68c4600aca56bccc4d62221'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:09.251315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a75c10f68c4600aca56bccc4d62221'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:09.252119 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-aa4f254d-50ae-463f-a871-d9733b5b948b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a75c10f68c4600aca56bccc4d62221'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:09.264494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1233/2507] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:08 2022] DELETE /identity/v3/users/73a75c10f68c4600aca56bccc4d62221 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:09.887986 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:09.890954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:09.917335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af7322-6d6d-4818-8619-c806795e80d5 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:09.975356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:09.992498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:09.993188 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:09.993574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:09.993955 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] PATH_INFO: `/v3/projects/df50f68455d54145a6e395f74b699068` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:09.996394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df50f68455d54145a6e395f74b699068)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:09.998133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:10.140926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df50f68455d54145a6e395f74b699068', 'request_id': 'req-588930ce-0884-4cd0-83d3-86d939b98ac9'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:10.147364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df50f68455d54145a6e395f74b699068', 'request_id': 'req-588930ce-0884-4cd0-83d3-86d939b98ac9'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:10.246379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df50f68455d54145a6e395f74b699068', 'request_id': 'req-588930ce-0884-4cd0-83d3-86d939b98ac9'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:10.264468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] The token cache is being invalidate because project df50f68455d54145a6e395f74b699068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:10.264468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:10.264468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:10.266727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1275/2508] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:09 2022] DELETE /identity/v3/projects/df50f68455d54145a6e395f74b699068 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:10.271862 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f142809-4c2a-4d0b-ab00-f485c16924dc None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:10.272926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f142809-4c2a-4d0b-ab00-f485c16924dc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:10.278682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f142809-4c2a-4d0b-ab00-f485c16924dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:10.278824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:10.278824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:10.305395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:10.329647 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-0f142809-4c2a-4d0b-ab00-f485c16924dc None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:10.331006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-0f142809-4c2a-4d0b-ab00-f485c16924dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:10.375417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588930ce-0884-4cd0-83d3-86d939b98ac9 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:10.382178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:10.384520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:10.384520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:10.384520 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] PATH_INFO: `/v3/users/c5e013bf52fa45dc8e4b65f183506cf7` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:10.385599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5e013bf52fa45dc8e4b65f183506cf7)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:10.388732 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:10.423665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1234/2509] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:10.434151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64145a7-5107-4900-82cd-fef4ecd45d2a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:10.435445 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64145a7-5107-4900-82cd-fef4ecd45d2a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:10.438993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64145a7-5107-4900-82cd-fef4ecd45d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:10.467196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f64145a7-5107-4900-82cd-fef4ecd45d2a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:10.468491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f64145a7-5107-4900-82cd-fef4ecd45d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:10.478150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1235/2510] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:10.482326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64145a7-5107-4900-82cd-fef4ecd45d2a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:10.493702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64145a7-5107-4900-82cd-fef4ecd45d2a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:10.500609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f6135-c91a-4bd8-8812-dc7b6766baac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:10.501510 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f6135-c91a-4bd8-8812-dc7b6766baac admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:10.502713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f6135-c91a-4bd8-8812-dc7b6766baac admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:10.503097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f6135-c91a-4bd8-8812-dc7b6766baac admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:10.504129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f6135-c91a-4bd8-8812-dc7b6766baac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:10.504709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f6135-c91a-4bd8-8812-dc7b6766baac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:10.506678 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f6135-c91a-4bd8-8812-dc7b6766baac admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:10.524857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1236/2511] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:36:10.539573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f6135-c91a-4bd8-8812-dc7b6766baac admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:10.551566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f6135-c91a-4bd8-8812-dc7b6766baac admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:10.556826 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf03388-960c-4ff9-9508-163207195eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:10.558200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf03388-960c-4ff9-9508-163207195eef admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:10.558501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf03388-960c-4ff9-9508-163207195eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:10.558784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf03388-960c-4ff9-9508-163207195eef admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:10.559465 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf03388-960c-4ff9-9508-163207195eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:10.562942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf03388-960c-4ff9-9508-163207195eef admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:10.568291 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5e013bf52fa45dc8e4b65f183506cf7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:10.611104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5e013bf52fa45dc8e4b65f183506cf7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:10.612800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5e013bf52fa45dc8e4b65f183506cf7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:10.620699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5e013bf52fa45dc8e4b65f183506cf7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:10.622515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:10.695217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1237/2512] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:10 2022] POST /identity/v3/projects => generated 357 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:10.699299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf03388-960c-4ff9-9508-163207195eef admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:10.755302 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e58cd222-4703-48fd-91f0-611f04573476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5e013bf52fa45dc8e4b65f183506cf7'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:10.765797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1276/2513] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:10 2022] DELETE /identity/v3/users/c5e013bf52fa45dc8e4b65f183506cf7 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:10.771402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1d0ba-dcd1-412d-bfa5-816421430ae6 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:10.772017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1d0ba-dcd1-412d-bfa5-816421430ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:10.772373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1d0ba-dcd1-412d-bfa5-816421430ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:10.793909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-38a1d0ba-dcd1-412d-bfa5-816421430ae6 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:10.795224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-38a1d0ba-dcd1-412d-bfa5-816421430ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:10.795476 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf03388-960c-4ff9-9508-163207195eef admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:10.873480 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3042009-f986-41e2-9ba5-c2d87fa51e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:10.875058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3042009-f986-41e2-9ba5-c2d87fa51e0f admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:10.875871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3042009-f986-41e2-9ba5-c2d87fa51e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:10.876661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3042009-f986-41e2-9ba5-c2d87fa51e0f admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:10.879648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3042009-f986-41e2-9ba5-c2d87fa51e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:10.883302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3042009-f986-41e2-9ba5-c2d87fa51e0f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:10.883550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1277/2514] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:10.893147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1d0ba-dcd1-412d-bfa5-816421430ae6 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:10.894980 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-38a1d0ba-dcd1-412d-bfa5-816421430ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:10.926411 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:10.957715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1d0ba-dcd1-412d-bfa5-816421430ae6 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:10.961853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f584fe-8b0f-488b-9ead-8eb4cc2bc1d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:10.962570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f584fe-8b0f-488b-9ead-8eb4cc2bc1d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:10.962861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f584fe-8b0f-488b-9ead-8eb4cc2bc1d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:10.963137 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f584fe-8b0f-488b-9ead-8eb4cc2bc1d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:10.973933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f584fe-8b0f-488b-9ead-8eb4cc2bc1d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:10.975511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f584fe-8b0f-488b-9ead-8eb4cc2bc1d5 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:10.986470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1278/2515] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:36:11.000278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1238/2516] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:10 2022] POST /identity/v3/users => generated 279 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:11.001337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f584fe-8b0f-488b-9ead-8eb4cc2bc1d5 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:11.005075 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f0f584fe-8b0f-488b-9ead-8eb4cc2bc1d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:11.018743 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f584fe-8b0f-488b-9ead-8eb4cc2bc1d5 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:11.029545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294fcf6-485c-4cfe-bd98-8a4abcceb29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:11.029545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294fcf6-485c-4cfe-bd98-8a4abcceb29c admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.029545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294fcf6-485c-4cfe-bd98-8a4abcceb29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.029545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294fcf6-485c-4cfe-bd98-8a4abcceb29c admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.029891 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294fcf6-485c-4cfe-bd98-8a4abcceb29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:11.029891 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294fcf6-485c-4cfe-bd98-8a4abcceb29c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:11.040861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1279/2517] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:11.045761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3042009-f986-41e2-9ba5-c2d87fa51e0f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:11.068102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3042009-f986-41e2-9ba5-c2d87fa51e0f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:11.084000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78805af-2f33-4735-a598-6a7501471394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:11.084680 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78805af-2f33-4735-a598-6a7501471394 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.085489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78805af-2f33-4735-a598-6a7501471394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.085489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78805af-2f33-4735-a598-6a7501471394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.086027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78805af-2f33-4735-a598-6a7501471394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:11.087654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78805af-2f33-4735-a598-6a7501471394 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:11.099093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1239/2518] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:11.109136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294fcf6-485c-4cfe-bd98-8a4abcceb29c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:11.123325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294fcf6-485c-4cfe-bd98-8a4abcceb29c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:11.128783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598a42d-a043-40b3-98d7-82c7ac142e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:11.129592 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598a42d-a043-40b3-98d7-82c7ac142e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.132482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598a42d-a043-40b3-98d7-82c7ac142e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.132482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598a42d-a043-40b3-98d7-82c7ac142e0f admin admin] PATH_INFO: `/v3/projects/19010298549c4be5b1946378e8c719eb/users/54e5615d17504e25836a2d4c43dc4022/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.147989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598a42d-a043-40b3-98d7-82c7ac142e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19010298549c4be5b1946378e8c719eb, user_id=54e5615d17504e25836a2d4c43dc4022, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:11.150957 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598a42d-a043-40b3-98d7-82c7ac142e0f admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:11.268341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1280/2519] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:11 2022] PUT /identity/v3/projects/19010298549c4be5b1946378e8c719eb/users/54e5615d17504e25836a2d4c43dc4022/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:11.279166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a264f13-814d-4403-98e9-930fc862e93e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.279700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a264f13-814d-4403-98e9-930fc862e93e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.279969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a264f13-814d-4403-98e9-930fc862e93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.310671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8a264f13-814d-4403-98e9-930fc862e93e None None] MFA Rules not processed for user `54e5615d17504e25836a2d4c43dc4022`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:11.312184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8a264f13-814d-4403-98e9-930fc862e93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:11.394780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1240/2520] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:11 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:11.399749 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598a42d-a043-40b3-98d7-82c7ac142e0f admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:11.465645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598a42d-a043-40b3-98d7-82c7ac142e0f admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:11.475428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c036b8-d099-4bbf-a412-a21ae2046299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:11.476077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c036b8-d099-4bbf-a412-a21ae2046299 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.476454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c036b8-d099-4bbf-a412-a21ae2046299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.476771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c036b8-d099-4bbf-a412-a21ae2046299 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.477481 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c036b8-d099-4bbf-a412-a21ae2046299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:11.479444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c036b8-d099-4bbf-a412-a21ae2046299 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:11.493490 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:11.567400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a264f13-814d-4403-98e9-930fc862e93e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:11.568886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8a264f13-814d-4403-98e9-930fc862e93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:11.589023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a264f13-814d-4403-98e9-930fc862e93e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:11.596928 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:11.605376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.605914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.606591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] PATH_INFO: `/v3/projects/d6b119ed685b4285b2c7ef69e3df53c4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.610025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b119ed685b4285b2c7ef69e3df53c4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:11.617804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1281/2521] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:11 2022] POST /identity/v3/projects => generated 355 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:11.618026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:11.619639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c036b8-d099-4bbf-a412-a21ae2046299 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:11.716586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c036b8-d099-4bbf-a412-a21ae2046299 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:11.721798 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8d7aa-de57-4b63-8e6b-02ab743ff1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:11.722455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8d7aa-de57-4b63-8e6b-02ab743ff1db admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.722739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8d7aa-de57-4b63-8e6b-02ab743ff1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.723059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8d7aa-de57-4b63-8e6b-02ab743ff1db admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.723744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8d7aa-de57-4b63-8e6b-02ab743ff1db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:11.725642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8d7aa-de57-4b63-8e6b-02ab743ff1db admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:11.802921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b119ed685b4285b2c7ef69e3df53c4', 'request_id': 'req-25f5fa5d-4a31-4c3a-9524-e00805992a2c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:11.812125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1282/2522] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:11 2022] POST /identity/v3/users => generated 284 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:11.815617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8d7aa-de57-4b63-8e6b-02ab743ff1db admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:11.821888 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b119ed685b4285b2c7ef69e3df53c4', 'request_id': 'req-25f5fa5d-4a31-4c3a-9524-e00805992a2c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:11.826186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b119ed685b4285b2c7ef69e3df53c4', 'request_id': 'req-25f5fa5d-4a31-4c3a-9524-e00805992a2c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:11.828145 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8d7aa-de57-4b63-8e6b-02ab743ff1db admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:11.885932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] The token cache is being invalidate because project d6b119ed685b4285b2c7ef69e3df53c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:11.886231 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:11.886942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-25f5fa5d-4a31-4c3a-9524-e00805992a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:11.888431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1241/2523] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:11 2022] DELETE /identity/v3/projects/d6b119ed685b4285b2c7ef69e3df53c4 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:11.893384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747c6a2-1ba5-4a5c-99c5-9b18bf958701 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.893683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747c6a2-1ba5-4a5c-99c5-9b18bf958701 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.898110 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747c6a2-1ba5-4a5c-99c5-9b18bf958701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.898230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1d19c-fb0d-4b13-85a6-e6340e39ce25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:11.898230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1d19c-fb0d-4b13-85a6-e6340e39ce25 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.898230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1d19c-fb0d-4b13-85a6-e6340e39ce25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.898230 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1d19c-fb0d-4b13-85a6-e6340e39ce25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.898950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1d19c-fb0d-4b13-85a6-e6340e39ce25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:11.899100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1d19c-fb0d-4b13-85a6-e6340e39ce25 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:11.909833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1283/2524] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:11.913762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1d19c-fb0d-4b13-85a6-e6340e39ce25 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:11.915529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1ec1d19c-fb0d-4b13-85a6-e6340e39ce25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:11.921040 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5747c6a2-1ba5-4a5c-99c5-9b18bf958701 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:11.921040 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5747c6a2-1ba5-4a5c-99c5-9b18bf958701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:11.928425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1242/2525] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:11.928575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1d19c-fb0d-4b13-85a6-e6340e39ce25 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:11.933261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019baae-2bba-40e8-823f-4bffd7eda533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:11.933963 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019baae-2bba-40e8-823f-4bffd7eda533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.934249 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019baae-2bba-40e8-823f-4bffd7eda533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.934540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019baae-2bba-40e8-823f-4bffd7eda533 admin admin] PATH_INFO: `/v3/projects/bd9bc28b775e4909937d42509b27bf93/users/1b0a928d9cfb4c318b7e18ad6247c2d3/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.936186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b36fc-e726-4630-9eda-ecac11f2c47d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.936575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b36fc-e726-4630-9eda-ecac11f2c47d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.936921 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b36fc-e726-4630-9eda-ecac11f2c47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.951081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019baae-2bba-40e8-823f-4bffd7eda533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9bc28b775e4909937d42509b27bf93, user_id=1b0a928d9cfb4c318b7e18ad6247c2d3, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:11.953842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019baae-2bba-40e8-823f-4bffd7eda533 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:11.958034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-366b36fc-e726-4630-9eda-ecac11f2c47d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:11.959430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-366b36fc-e726-4630-9eda-ecac11f2c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:11.968396 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1243/2526] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:11.976808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b36fc-e726-4630-9eda-ecac11f2c47d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:11.988305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b36fc-e726-4630-9eda-ecac11f2c47d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:11.993490 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb27cee-65c4-4888-a02d-5ed2f6d6da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:11.994317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27cee-65c4-4888-a02d-5ed2f6d6da8b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:11.994812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27cee-65c4-4888-a02d-5ed2f6d6da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:11.995317 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27cee-65c4-4888-a02d-5ed2f6d6da8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:11.996186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb27cee-65c4-4888-a02d-5ed2f6d6da8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:11.996494 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb27cee-65c4-4888-a02d-5ed2f6d6da8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:11.998282 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb27cee-65c4-4888-a02d-5ed2f6d6da8b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.018763 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1244/2527] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:36:12.036936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb27cee-65c4-4888-a02d-5ed2f6d6da8b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:12.036936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb27cee-65c4-4888-a02d-5ed2f6d6da8b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:12.036936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532828d1-4526-4460-95ad-59601b6bc23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:12.036936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532828d1-4526-4460-95ad-59601b6bc23f admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.037403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532828d1-4526-4460-95ad-59601b6bc23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.037403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532828d1-4526-4460-95ad-59601b6bc23f admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.037403 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532828d1-4526-4460-95ad-59601b6bc23f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:12.038170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1284/2528] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:11 2022] PUT /identity/v3/projects/bd9bc28b775e4909937d42509b27bf93/users/1b0a928d9cfb4c318b7e18ad6247c2d3/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:12.039179 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532828d1-4526-4460-95ad-59601b6bc23f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.042063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019baae-2bba-40e8-823f-4bffd7eda533 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:12.102036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:12.117532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019baae-2bba-40e8-823f-4bffd7eda533 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:12.122339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83a9eb-e91c-40be-8834-2d2464dd1f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:12.122992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83a9eb-e91c-40be-8834-2d2464dd1f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.123316 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83a9eb-e91c-40be-8834-2d2464dd1f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.123593 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83a9eb-e91c-40be-8834-2d2464dd1f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.124498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83a9eb-e91c-40be-8834-2d2464dd1f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:12.125942 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83a9eb-e91c-40be-8834-2d2464dd1f21 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.136378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1285/2529] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:12.140598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83a9eb-e91c-40be-8834-2d2464dd1f21 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:12.150563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83a9eb-e91c-40be-8834-2d2464dd1f21 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:12.155243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d45c7-c4bd-47b9-9c2e-f84525023e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:12.155899 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d45c7-c4bd-47b9-9c2e-f84525023e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.156189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d45c7-c4bd-47b9-9c2e-f84525023e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.156468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d45c7-c4bd-47b9-9c2e-f84525023e03 admin admin] PATH_INFO: `/v3/projects/bd9bc28b775e4909937d42509b27bf93/users/1b0a928d9cfb4c318b7e18ad6247c2d3/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.158512 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d45c7-c4bd-47b9-9c2e-f84525023e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9bc28b775e4909937d42509b27bf93, user_id=1b0a928d9cfb4c318b7e18ad6247c2d3, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:12.161827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d45c7-c4bd-47b9-9c2e-f84525023e03 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.171296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1245/2530] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:12 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:12.172443 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532828d1-4526-4460-95ad-59601b6bc23f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:12.173573 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1286/2531] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:12 2022] PUT /identity/v3/projects/bd9bc28b775e4909937d42509b27bf93/users/1b0a928d9cfb4c318b7e18ad6247c2d3/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:12.183673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:12.184925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3097d-eee6-414f-b81b-2517f4bbcfb4 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.185273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3097d-eee6-414f-b81b-2517f4bbcfb4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.185585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3097d-eee6-414f-b81b-2517f4bbcfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.220580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-99a3097d-eee6-414f-b81b-2517f4bbcfb4 None None] MFA Rules not processed for user `1b0a928d9cfb4c318b7e18ad6247c2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:12.221803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-99a3097d-eee6-414f-b81b-2517f4bbcfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:12.249049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532828d1-4526-4460-95ad-59601b6bc23f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:12.253980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae7bfb-23bc-482f-bb5c-a42862de049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:12.254772 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae7bfb-23bc-482f-bb5c-a42862de049f admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.255104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae7bfb-23bc-482f-bb5c-a42862de049f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.255343 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae7bfb-23bc-482f-bb5c-a42862de049f admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.256794 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae7bfb-23bc-482f-bb5c-a42862de049f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:12.259086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae7bfb-23bc-482f-bb5c-a42862de049f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.307864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1287/2532] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:12 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:12.316675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3097d-eee6-414f-b81b-2517f4bbcfb4 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:12.318239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-99a3097d-eee6-414f-b81b-2517f4bbcfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:12.361759 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1246/2533] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:12 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:12.366836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae7bfb-23bc-482f-bb5c-a42862de049f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:12.377365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae7bfb-23bc-482f-bb5c-a42862de049f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:12.382436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3097d-eee6-414f-b81b-2517f4bbcfb4 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:12.382588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa053d47-e038-455c-83c0-7d95b66ff2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:12.383137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa053d47-e038-455c-83c0-7d95b66ff2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.383420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa053d47-e038-455c-83c0-7d95b66ff2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.383705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa053d47-e038-455c-83c0-7d95b66ff2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.384412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa053d47-e038-455c-83c0-7d95b66ff2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:12.386072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa053d47-e038-455c-83c0-7d95b66ff2a6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.386329 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd6959-c96b-4df1-9c81-a1cd7922268f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:12.387094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd6959-c96b-4df1-9c81-a1cd7922268f service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.387655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd6959-c96b-4df1-9c81-a1cd7922268f service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.387945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd6959-c96b-4df1-9c81-a1cd7922268f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.396728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1247/2534] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:12.399201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd6959-c96b-4df1-9c81-a1cd7922268f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:12.403966 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd6959-c96b-4df1-9c81-a1cd7922268f service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.407127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa053d47-e038-455c-83c0-7d95b66ff2a6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:12.412326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1288/2535] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:12 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:12.419245 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa053d47-e038-455c-83c0-7d95b66ff2a6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:12.424078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a46b44-0005-40a2-b794-e8d9b5553058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:12.424794 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a46b44-0005-40a2-b794-e8d9b5553058 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.424993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a46b44-0005-40a2-b794-e8d9b5553058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.425363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a46b44-0005-40a2-b794-e8d9b5553058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.426667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a46b44-0005-40a2-b794-e8d9b5553058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:12.432219 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a46b44-0005-40a2-b794-e8d9b5553058 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.442073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1248/2536] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:12.446353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd6959-c96b-4df1-9c81-a1cd7922268f service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:12.459762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd6959-c96b-4df1-9c81-a1cd7922268f service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:12.464958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50f6e-106b-43a8-a272-a05838407d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:12.465673 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50f6e-106b-43a8-a272-a05838407d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.465962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50f6e-106b-43a8-a272-a05838407d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.466262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50f6e-106b-43a8-a272-a05838407d22 admin admin] PATH_INFO: `/v3/projects/a420b01a5b8348b290f1ad5c9852f524/users/6b3e6241a9f247279e72d81718f55b3d/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.482274 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d50f6e-106b-43a8-a272-a05838407d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a420b01a5b8348b290f1ad5c9852f524, user_id=6b3e6241a9f247279e72d81718f55b3d, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:12.484949 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d50f6e-106b-43a8-a272-a05838407d22 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.577733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1289/2537] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:12 2022] PUT /identity/v3/projects/a420b01a5b8348b290f1ad5c9852f524/users/6b3e6241a9f247279e72d81718f55b3d/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:12.590357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d1849-aa73-4522-829b-6d3c1a89156d None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.590652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d1849-aa73-4522-829b-6d3c1a89156d None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.590940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d1849-aa73-4522-829b-6d3c1a89156d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.620383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-533d1849-aa73-4522-829b-6d3c1a89156d None None] MFA Rules not processed for user `6b3e6241a9f247279e72d81718f55b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:12.621714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-533d1849-aa73-4522-829b-6d3c1a89156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:12.642662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50f6e-106b-43a8-a272-a05838407d22 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:12.681521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:12.692130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1249/2538] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:12.703905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d1849-aa73-4522-829b-6d3c1a89156d None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:12.708461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-533d1849-aa73-4522-829b-6d3c1a89156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:12.709845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50f6e-106b-43a8-a272-a05838407d22 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:12.715073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcbf4a-627a-4b9b-980c-962130fc2aa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:12.715872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcbf4a-627a-4b9b-980c-962130fc2aa1 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.716217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcbf4a-627a-4b9b-980c-962130fc2aa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.716694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcbf4a-627a-4b9b-980c-962130fc2aa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.718638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2afcbf4a-627a-4b9b-980c-962130fc2aa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:12.781248 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:12.783225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcbf4a-627a-4b9b-980c-962130fc2aa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:12.785267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcbf4a-627a-4b9b-980c-962130fc2aa1 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.797256 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1290/2539] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:12 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:12.823666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d1849-aa73-4522-829b-6d3c1a89156d None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:12.829861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b802c4f-d359-46ce-a078-4131f2880f84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:12.830051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b802c4f-d359-46ce-a078-4131f2880f84 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:12.830344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b802c4f-d359-46ce-a078-4131f2880f84 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:12.830628 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b802c4f-d359-46ce-a078-4131f2880f84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:12.839665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b802c4f-d359-46ce-a078-4131f2880f84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:12.841214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b802c4f-d359-46ce-a078-4131f2880f84 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:12.851896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1250/2540] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:36:14.894244 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcbf4a-627a-4b9b-980c-962130fc2aa1 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:14.896431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2afcbf4a-627a-4b9b-980c-962130fc2aa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:14.962674 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcbf4a-627a-4b9b-980c-962130fc2aa1 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:14.967268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd54072-c55e-4876-9a5f-0558693d2b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:14.968122 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd54072-c55e-4876-9a5f-0558693d2b51 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:14.968298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd54072-c55e-4876-9a5f-0558693d2b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:14.968727 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd54072-c55e-4876-9a5f-0558693d2b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:14.971128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2dd54072-c55e-4876-9a5f-0558693d2b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:15.035580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd54072-c55e-4876-9a5f-0558693d2b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:15.037027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd54072-c55e-4876-9a5f-0558693d2b51 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:15.046961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1291/2541] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:15.254993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b802c4f-d359-46ce-a078-4131f2880f84 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:15.256579 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4b802c4f-d359-46ce-a078-4131f2880f84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:15.355041 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b802c4f-d359-46ce-a078-4131f2880f84 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:15.360389 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53bee3-9151-4b5a-83e6-209b8b313500 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:15.361173 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53bee3-9151-4b5a-83e6-209b8b313500 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:15.361564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53bee3-9151-4b5a-83e6-209b8b313500 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:15.361854 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53bee3-9151-4b5a-83e6-209b8b313500 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:15.367531 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ea53bee3-9151-4b5a-83e6-209b8b313500 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:15.380430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53bee3-9151-4b5a-83e6-209b8b313500 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:15.382187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53bee3-9151-4b5a-83e6-209b8b313500 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:15.392824 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1251/2542] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:16.252456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd54072-c55e-4876-9a5f-0558693d2b51 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:16.261545 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd54072-c55e-4876-9a5f-0558693d2b51 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:16.266373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:16.267017 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:16.267330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:16.267579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] PATH_INFO: `/v3/users/6b3e6241a9f247279e72d81718f55b3d` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:16.268741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b3e6241a9f247279e72d81718f55b3d)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:16.270803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:16.481710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:16.495033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b3e6241a9f247279e72d81718f55b3d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:16.507744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b3e6241a9f247279e72d81718f55b3d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:16.508481 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b3e6241a9f247279e72d81718f55b3d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:16.515472 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b3e6241a9f247279e72d81718f55b3d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:16.598413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b3e6241a9f247279e72d81718f55b3d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:16.604936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1292/2543] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:16 2022] DELETE /identity/v3/users/6b3e6241a9f247279e72d81718f55b3d => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:16.609971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a02bf7-c7f9-421e-addf-e75c60b6a8af None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:16.609971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a02bf7-c7f9-421e-addf-e75c60b6a8af None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:16.610565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a02bf7-c7f9-421e-addf-e75c60b6a8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:16.631741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a9a02bf7-c7f9-421e-addf-e75c60b6a8af None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:16.638858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a9a02bf7-c7f9-421e-addf-e75c60b6a8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:16.709007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1252/2544] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:16.720736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:16.722764 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:16.842613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4782b08-cec9-478e-91bc-17d827cdf16d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:16.847976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456210de-89a7-4219-a703-4be01c293be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:16.848801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456210de-89a7-4219-a703-4be01c293be9 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:16.849183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456210de-89a7-4219-a703-4be01c293be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:16.849677 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456210de-89a7-4219-a703-4be01c293be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:16.863588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456210de-89a7-4219-a703-4be01c293be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:16.865187 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456210de-89a7-4219-a703-4be01c293be9 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:16.878149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1293/2545] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:17.538461 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a02bf7-c7f9-421e-addf-e75c60b6a8af None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:17.539908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a9a02bf7-c7f9-421e-addf-e75c60b6a8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:17.552293 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a02bf7-c7f9-421e-addf-e75c60b6a8af None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:17.557188 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:17.557912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:17.558193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:17.558478 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] PATH_INFO: `/v3/projects/a420b01a5b8348b290f1ad5c9852f524` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:17.559656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a420b01a5b8348b290f1ad5c9852f524)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:17.561648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:17.682291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:17.698925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a420b01a5b8348b290f1ad5c9852f524', 'request_id': 'req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:17.711308 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a420b01a5b8348b290f1ad5c9852f524', 'request_id': 'req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:17.725833 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a420b01a5b8348b290f1ad5c9852f524', 'request_id': 'req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:17.790074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] The token cache is being invalidate because project a420b01a5b8348b290f1ad5c9852f524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:17.790074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:17.791797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ebd67dd9-1b3b-4a0c-b6ae-d6f77943136a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:17.792468 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1253/2546] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:17 2022] DELETE /identity/v3/projects/a420b01a5b8348b290f1ad5c9852f524 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:17.799092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a407a-8a2b-4c68-be1a-aef0add2e104 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:17.801100 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a407a-8a2b-4c68-be1a-aef0add2e104 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:17.802358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a407a-8a2b-4c68-be1a-aef0add2e104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:17.851658 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-831a407a-8a2b-4c68-be1a-aef0add2e104 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:17.853002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-831a407a-8a2b-4c68-be1a-aef0add2e104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:17.924898 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1294/2547] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:17.934286 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c6c89-e6ed-42fb-a9f3-76d15acaf693 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:17.934661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c6c89-e6ed-42fb-a9f3-76d15acaf693 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:17.935061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c6c89-e6ed-42fb-a9f3-76d15acaf693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:17.960410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-f18c6c89-e6ed-42fb-a9f3-76d15acaf693 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:17.961755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-f18c6c89-e6ed-42fb-a9f3-76d15acaf693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:17.974589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1254/2548] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:17.982707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a407a-8a2b-4c68-be1a-aef0add2e104 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:18.000087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a407a-8a2b-4c68-be1a-aef0add2e104 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:18.017004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d42cf-3aeb-402a-ba88-0ff4e15b9bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:18.017676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d42cf-3aeb-402a-ba88-0ff4e15b9bee admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:18.017962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d42cf-3aeb-402a-ba88-0ff4e15b9bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:18.018243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d42cf-3aeb-402a-ba88-0ff4e15b9bee admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:18.019639 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d42cf-3aeb-402a-ba88-0ff4e15b9bee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:18.019962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d42cf-3aeb-402a-ba88-0ff4e15b9bee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:18.021704 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d42cf-3aeb-402a-ba88-0ff4e15b9bee admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:18.034879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1295/2549] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:36:18.040243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c6c89-e6ed-42fb-a9f3-76d15acaf693 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:18.070414 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c6c89-e6ed-42fb-a9f3-76d15acaf693 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:18.095985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fc6b3-1f77-49d1-b8c1-9d5bbb735d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:18.095985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fc6b3-1f77-49d1-b8c1-9d5bbb735d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:18.095985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fc6b3-1f77-49d1-b8c1-9d5bbb735d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:18.095985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fc6b3-1f77-49d1-b8c1-9d5bbb735d93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:18.096506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fc6b3-1f77-49d1-b8c1-9d5bbb735d93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:18.096506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fc6b3-1f77-49d1-b8c1-9d5bbb735d93 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:18.262988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1255/2550] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:18 2022] POST /identity/v3/projects => generated 403 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:18.265210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d42cf-3aeb-402a-ba88-0ff4e15b9bee admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:18.278251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d42cf-3aeb-402a-ba88-0ff4e15b9bee admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:18.345622 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:18.387502 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919817f1-f4c3-4d45-be2f-bfed17a1671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:18.388569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919817f1-f4c3-4d45-be2f-bfed17a1671d admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:18.389245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919817f1-f4c3-4d45-be2f-bfed17a1671d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:18.389693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919817f1-f4c3-4d45-be2f-bfed17a1671d admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:18.391000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919817f1-f4c3-4d45-be2f-bfed17a1671d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:18.395346 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919817f1-f4c3-4d45-be2f-bfed17a1671d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:18.534752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1296/2551] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:18 2022] POST /identity/v3/users => generated 302 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:18.537128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fc6b3-1f77-49d1-b8c1-9d5bbb735d93 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:18.554734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fc6b3-1f77-49d1-b8c1-9d5bbb735d93 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:18.562206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51bb81-7e65-4fc6-a9ef-d38aacdf29ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:18.562993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51bb81-7e65-4fc6-a9ef-d38aacdf29ae admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:18.563374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51bb81-7e65-4fc6-a9ef-d38aacdf29ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:18.563766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51bb81-7e65-4fc6-a9ef-d38aacdf29ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:18.564729 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51bb81-7e65-4fc6-a9ef-d38aacdf29ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:18.569232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51bb81-7e65-4fc6-a9ef-d38aacdf29ae admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:18.585362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1256/2552] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:18.589183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919817f1-f4c3-4d45-be2f-bfed17a1671d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:18.607804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919817f1-f4c3-4d45-be2f-bfed17a1671d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:18.627514 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863abc09-f8dd-41c1-a8b9-eb7a4e773c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:18.630428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863abc09-f8dd-41c1-a8b9-eb7a4e773c38 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:18.630985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863abc09-f8dd-41c1-a8b9-eb7a4e773c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:18.631337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863abc09-f8dd-41c1-a8b9-eb7a4e773c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:18.633358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863abc09-f8dd-41c1-a8b9-eb7a4e773c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:18.639444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863abc09-f8dd-41c1-a8b9-eb7a4e773c38 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:18.651361 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1297/2553] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:18.656724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51bb81-7e65-4fc6-a9ef-d38aacdf29ae admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:18.692398 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51bb81-7e65-4fc6-a9ef-d38aacdf29ae admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:18.702716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad884eb-916a-47d1-8d0d-aee6ba5421a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:18.702716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad884eb-916a-47d1-8d0d-aee6ba5421a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:18.702716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad884eb-916a-47d1-8d0d-aee6ba5421a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:18.702716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad884eb-916a-47d1-8d0d-aee6ba5421a6 admin admin] PATH_INFO: `/v3/projects/c2e35bf1f4f74e8d979bae3f767f4757/users/e32a1422fd0e4becb5db8812bc2cea47/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:18.726374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad884eb-916a-47d1-8d0d-aee6ba5421a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e35bf1f4f74e8d979bae3f767f4757, user_id=e32a1422fd0e4becb5db8812bc2cea47, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:18.730246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad884eb-916a-47d1-8d0d-aee6ba5421a6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:18.811942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1257/2554] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:18 2022] PUT /identity/v3/projects/c2e35bf1f4f74e8d979bae3f767f4757/users/e32a1422fd0e4becb5db8812bc2cea47/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:18.821856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af201de-0e0f-438b-a152-954702280661 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:18.822402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af201de-0e0f-438b-a152-954702280661 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:18.822525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af201de-0e0f-438b-a152-954702280661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:18.858661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-3af201de-0e0f-438b-a152-954702280661 None None] MFA Rules not processed for user `e32a1422fd0e4becb5db8812bc2cea47`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:18.859902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3af201de-0e0f-438b-a152-954702280661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:18.961394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1298/2555] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:18.964711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad884eb-916a-47d1-8d0d-aee6ba5421a6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:19.040625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad884eb-916a-47d1-8d0d-aee6ba5421a6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:19.047469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b22c7-5f92-4018-b2e0-30ac585595db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:19.048116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b22c7-5f92-4018-b2e0-30ac585595db admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:19.048409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b22c7-5f92-4018-b2e0-30ac585595db admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:19.048695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b22c7-5f92-4018-b2e0-30ac585595db admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:19.049426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b22c7-5f92-4018-b2e0-30ac585595db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:19.051725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b22c7-5f92-4018-b2e0-30ac585595db admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:19.156544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1258/2556] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:18 2022] POST /identity/v3/projects => generated 403 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:19.161603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af201de-0e0f-438b-a152-954702280661 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:19.173661 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af201de-0e0f-438b-a152-954702280661 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:19.232350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80d7e4-d4b0-4092-8ad0-b4483b6503e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:19.232931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80d7e4-d4b0-4092-8ad0-b4483b6503e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:19.233211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80d7e4-d4b0-4092-8ad0-b4483b6503e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:19.233531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80d7e4-d4b0-4092-8ad0-b4483b6503e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:19.234215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80d7e4-d4b0-4092-8ad0-b4483b6503e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:19.236140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80d7e4-d4b0-4092-8ad0-b4483b6503e5 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:19.316193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b22c7-5f92-4018-b2e0-30ac585595db admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:19.318160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-088b22c7-5f92-4018-b2e0-30ac585595db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:19.337214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b22c7-5f92-4018-b2e0-30ac585595db admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:19.337681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:19.338347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:19.338640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:19.338928 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] PATH_INFO: `/v3/users/6d4528589ffa478689e905d47e4c37ee` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:19.340216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d4528589ffa478689e905d47e4c37ee)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:19.342211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:19.371845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1299/2557] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:19 2022] POST /identity/v3/users => generated 308 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:19.376152 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80d7e4-d4b0-4092-8ad0-b4483b6503e5 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:19.387785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80d7e4-d4b0-4092-8ad0-b4483b6503e5 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:19.392785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588aba4-0d83-476b-892a-6e177ff7e8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:19.393509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588aba4-0d83-476b-892a-6e177ff7e8db admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:19.394369 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588aba4-0d83-476b-892a-6e177ff7e8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:19.394744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588aba4-0d83-476b-892a-6e177ff7e8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:19.395555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588aba4-0d83-476b-892a-6e177ff7e8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:19.397240 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588aba4-0d83-476b-892a-6e177ff7e8db admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:19.408370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1300/2558] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:19.413844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588aba4-0d83-476b-892a-6e177ff7e8db admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:19.428620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588aba4-0d83-476b-892a-6e177ff7e8db admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:19.435427 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007f884-7af5-4199-b5d3-0f901f0cfb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:19.436395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007f884-7af5-4199-b5d3-0f901f0cfb0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:19.436828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007f884-7af5-4199-b5d3-0f901f0cfb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:19.437238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007f884-7af5-4199-b5d3-0f901f0cfb0b admin admin] PATH_INFO: `/v3/projects/62aa01a5827e4c03b43dfceba5f27e11/users/e2bfca12470f48fcb7567cc44e01a5b8/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:19.454218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007f884-7af5-4199-b5d3-0f901f0cfb0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=62aa01a5827e4c03b43dfceba5f27e11, user_id=e2bfca12470f48fcb7567cc44e01a5b8, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:19.458340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007f884-7af5-4199-b5d3-0f901f0cfb0b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:19.520914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:19.536056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d4528589ffa478689e905d47e4c37ee'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:19.549037 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d4528589ffa478689e905d47e4c37ee'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:19.554054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1301/2559] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:19 2022] PUT /identity/v3/projects/62aa01a5827e4c03b43dfceba5f27e11/users/e2bfca12470f48fcb7567cc44e01a5b8/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:19.558055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007f884-7af5-4199-b5d3-0f901f0cfb0b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:19.558496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d4528589ffa478689e905d47e4c37ee'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:19.565774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:19.635391 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007f884-7af5-4199-b5d3-0f901f0cfb0b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:19.640189 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da889903-81f9-47b8-825f-f863287bea1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:19.640806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da889903-81f9-47b8-825f-f863287bea1b admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:19.641099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da889903-81f9-47b8-825f-f863287bea1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:19.641419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da889903-81f9-47b8-825f-f863287bea1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:19.642130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da889903-81f9-47b8-825f-f863287bea1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:19.643856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da889903-81f9-47b8-825f-f863287bea1b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:19.651147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d4528589ffa478689e905d47e4c37ee'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:19.651884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-afe20011-7a0c-4459-b3ad-12ec2bc21cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d4528589ffa478689e905d47e4c37ee'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:19.655395 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1302/2560] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:19.659471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da889903-81f9-47b8-825f-f863287bea1b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:19.660545 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1259/2561] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:19 2022] DELETE /identity/v3/users/6d4528589ffa478689e905d47e4c37ee => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:19.661558 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-da889903-81f9-47b8-825f-f863287bea1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:19.677010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da889903-81f9-47b8-825f-f863287bea1b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:19.740925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cc540-fdce-4c02-801f-3d461272831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:19.741659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cc540-fdce-4c02-801f-3d461272831c admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:19.741989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cc540-fdce-4c02-801f-3d461272831c admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:19.742872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cc540-fdce-4c02-801f-3d461272831c admin admin] PATH_INFO: `/v3/projects/62aa01a5827e4c03b43dfceba5f27e11/users/e2bfca12470f48fcb7567cc44e01a5b8/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:19.744810 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cc540-fdce-4c02-801f-3d461272831c admin admin] RBAC: Authorizing `identity:create_grant(project_id=62aa01a5827e4c03b43dfceba5f27e11, user_id=e2bfca12470f48fcb7567cc44e01a5b8, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:19.748290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cc540-fdce-4c02-801f-3d461272831c admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:19.762935 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1303/2562] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:19 2022] PUT /identity/v3/projects/62aa01a5827e4c03b43dfceba5f27e11/users/e2bfca12470f48fcb7567cc44e01a5b8/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:19.775395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64b5e-b3a4-4747-b96a-de5c6b136605 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:19.776298 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64b5e-b3a4-4747-b96a-de5c6b136605 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:19.776509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64b5e-b3a4-4747-b96a-de5c6b136605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:19.820598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-43a64b5e-b3a4-4747-b96a-de5c6b136605 None None] MFA Rules not processed for user `e2bfca12470f48fcb7567cc44e01a5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:19.822218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-43a64b5e-b3a4-4747-b96a-de5c6b136605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:19.911455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cc540-fdce-4c02-801f-3d461272831c admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:19.913382 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-329cc540-fdce-4c02-801f-3d461272831c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:19.913809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1260/2563] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:19 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:19.924938 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a64b5e-b3a4-4747-b96a-de5c6b136605 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:19.926458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-43a64b5e-b3a4-4747-b96a-de5c6b136605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:19.994509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a64b5e-b3a4-4747-b96a-de5c6b136605 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:19.999924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c8603-061d-4f63-b26d-7528d8f20c6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:20.000638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cc540-fdce-4c02-801f-3d461272831c admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:20.000925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c8603-061d-4f63-b26d-7528d8f20c6b service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.001371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c8603-061d-4f63-b26d-7528d8f20c6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.001891 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c8603-061d-4f63-b26d-7528d8f20c6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.008011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:20.009958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.010434 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.011423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] PATH_INFO: `/v3/projects/a164b3cd6f444e17ae7ada0e30e62f54` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.013163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c8603-061d-4f63-b26d-7528d8f20c6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:20.013897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a164b3cd6f444e17ae7ada0e30e62f54)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:20.014672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c8603-061d-4f63-b26d-7528d8f20c6b service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:20.015977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:20.024965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1261/2564] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:19 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:36:20.172941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:20.186281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a164b3cd6f444e17ae7ada0e30e62f54', 'request_id': 'req-cd3a2bfa-dd13-49dd-a337-156da97d0d62'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:20.193087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a164b3cd6f444e17ae7ada0e30e62f54', 'request_id': 'req-cd3a2bfa-dd13-49dd-a337-156da97d0d62'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:20.199238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c8603-061d-4f63-b26d-7528d8f20c6b service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:20.211310 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c8603-061d-4f63-b26d-7528d8f20c6b service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:20.246287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:20.270913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:20.271722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.271948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.272304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] PATH_INFO: `/v3/users/e32a1422fd0e4becb5db8812bc2cea47` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.274076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32a1422fd0e4becb5db8812bc2cea47)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:20.276066 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:20.308356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a164b3cd6f444e17ae7ada0e30e62f54', 'request_id': 'req-cd3a2bfa-dd13-49dd-a337-156da97d0d62'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:20.323666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] The token cache is being invalidate because project a164b3cd6f444e17ae7ada0e30e62f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:20.323973 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:20.324652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-cd3a2bfa-dd13-49dd-a337-156da97d0d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:20.327128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1304/2565] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:19 2022] DELETE /identity/v3/projects/a164b3cd6f444e17ae7ada0e30e62f54 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:20.332112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9dbb6-9236-4462-9afa-f3084763f4e1 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.332498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9dbb6-9236-4462-9afa-f3084763f4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.332795 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9dbb6-9236-4462-9afa-f3084763f4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.357757 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-27c9dbb6-9236-4462-9afa-f3084763f4e1 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:20.358904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-27c9dbb6-9236-4462-9afa-f3084763f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:20.369245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1305/2566] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:20.377517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d879-4df3-477c-b7cc-f6ef58348633 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.377818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d879-4df3-477c-b7cc-f6ef58348633 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.378127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d879-4df3-477c-b7cc-f6ef58348633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.399783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-be76d879-4df3-477c-b7cc-f6ef58348633 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:20.401024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-be76d879-4df3-477c-b7cc-f6ef58348633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:20.411861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1306/2567] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:20.416823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d879-4df3-477c-b7cc-f6ef58348633 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:20.427326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d879-4df3-477c-b7cc-f6ef58348633 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:20.433143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bd4e9-421a-430c-a68c-c6847375ca18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:20.434396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bd4e9-421a-430c-a68c-c6847375ca18 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.434982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bd4e9-421a-430c-a68c-c6847375ca18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.435509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bd4e9-421a-430c-a68c-c6847375ca18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.436842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9bd4e9-421a-430c-a68c-c6847375ca18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:20.437834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9bd4e9-421a-430c-a68c-c6847375ca18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:20.439404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9bd4e9-421a-430c-a68c-c6847375ca18 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:20.456974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1307/2568] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:36:20.462425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bd4e9-421a-430c-a68c-c6847375ca18 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:20.474365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bd4e9-421a-430c-a68c-c6847375ca18 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:20.479554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc2add-88d6-403f-befa-ff9e980b60dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:20.480283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc2add-88d6-403f-befa-ff9e980b60dc admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.480565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc2add-88d6-403f-befa-ff9e980b60dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.480845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc2add-88d6-403f-befa-ff9e980b60dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.481688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc2add-88d6-403f-befa-ff9e980b60dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:20.484524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc2add-88d6-403f-befa-ff9e980b60dc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:20.512508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32a1422fd0e4becb5db8812bc2cea47'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:20.528332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32a1422fd0e4becb5db8812bc2cea47'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:20.538269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32a1422fd0e4becb5db8812bc2cea47'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:20.591138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1308/2569] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:20 2022] POST /identity/v3/projects => generated 405 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:20.595409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc2add-88d6-403f-befa-ff9e980b60dc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:20.626657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32a1422fd0e4becb5db8812bc2cea47'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:20.627340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83d15e3a-b103-4870-ada8-a7e9ec87e8e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32a1422fd0e4becb5db8812bc2cea47'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:20.638222 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1262/2570] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:20 2022] DELETE /identity/v3/users/e32a1422fd0e4becb5db8812bc2cea47 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:20.645935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ed925-e69a-446e-bf3f-db8e93969454 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.646289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ed925-e69a-446e-bf3f-db8e93969454 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.646809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ed925-e69a-446e-bf3f-db8e93969454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.668020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc2add-88d6-403f-befa-ff9e980b60dc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:20.672618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a18ed925-e69a-446e-bf3f-db8e93969454 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:20.673931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a18ed925-e69a-446e-bf3f-db8e93969454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:20.734549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee48b7-5953-40ec-9201-1c4ddc42cbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:20.735349 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee48b7-5953-40ec-9201-1c4ddc42cbe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.735710 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee48b7-5953-40ec-9201-1c4ddc42cbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.736077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee48b7-5953-40ec-9201-1c4ddc42cbe7 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.736760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee48b7-5953-40ec-9201-1c4ddc42cbe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:20.739613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee48b7-5953-40ec-9201-1c4ddc42cbe7 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:20.748124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1263/2571] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:20.760002 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ed925-e69a-446e-bf3f-db8e93969454 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:20.761994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a18ed925-e69a-446e-bf3f-db8e93969454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:20.853416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ed925-e69a-446e-bf3f-db8e93969454 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:20.858639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273f284-e16f-446c-af7c-6a233dfaabf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:20.859415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273f284-e16f-446c-af7c-6a233dfaabf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.859734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273f284-e16f-446c-af7c-6a233dfaabf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.860241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273f284-e16f-446c-af7c-6a233dfaabf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.873746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273f284-e16f-446c-af7c-6a233dfaabf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:20.876059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1309/2572] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:20 2022] POST /identity/v3/users => generated 303 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:20.882004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273f284-e16f-446c-af7c-6a233dfaabf2 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:20.882148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee48b7-5953-40ec-9201-1c4ddc42cbe7 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:20.882842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-61ee48b7-5953-40ec-9201-1c4ddc42cbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:20.883605 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:20.891129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1264/2573] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:36:20.897663 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee48b7-5953-40ec-9201-1c4ddc42cbe7 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:20.902797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b347584-91bf-477e-ba95-9e56e77ad218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:20.903435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b347584-91bf-477e-ba95-9e56e77ad218 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.903717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b347584-91bf-477e-ba95-9e56e77ad218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.904009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b347584-91bf-477e-ba95-9e56e77ad218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.904766 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b347584-91bf-477e-ba95-9e56e77ad218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:20.906559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b347584-91bf-477e-ba95-9e56e77ad218 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:20.913708 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:20.922193 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1310/2574] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:20.923870 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273f284-e16f-446c-af7c-6a233dfaabf2 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:20.935885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273f284-e16f-446c-af7c-6a233dfaabf2 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:20.941263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b811b-ef52-44eb-835b-fb0c2bd36424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:20.942028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b811b-ef52-44eb-835b-fb0c2bd36424 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.942316 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b811b-ef52-44eb-835b-fb0c2bd36424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.942599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b811b-ef52-44eb-835b-fb0c2bd36424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:20.943314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b811b-ef52-44eb-835b-fb0c2bd36424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:20.944960 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b811b-ef52-44eb-835b-fb0c2bd36424 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:20.959834 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1265/2575] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:20.964313 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b347584-91bf-477e-ba95-9e56e77ad218 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:20.976894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b347584-91bf-477e-ba95-9e56e77ad218 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:20.983251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d96851-2705-4f28-b003-7209c42fec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:20.985196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d96851-2705-4f28-b003-7209c42fec9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:20.985196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d96851-2705-4f28-b003-7209c42fec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:20.985196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d96851-2705-4f28-b003-7209c42fec9a admin admin] PATH_INFO: `/v3/projects/363aa7c3840443ae8d314eb69d382045/users/caf7d2139ef24629b23f01c08b08a1a7/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:21.001765 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d96851-2705-4f28-b003-7209c42fec9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=363aa7c3840443ae8d314eb69d382045, user_id=caf7d2139ef24629b23f01c08b08a1a7, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:21.005003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d96851-2705-4f28-b003-7209c42fec9a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:21.114745 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1311/2576] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:20 2022] PUT /identity/v3/projects/363aa7c3840443ae8d314eb69d382045/users/caf7d2139ef24629b23f01c08b08a1a7/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:21.120107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9f2f3-4c76-4d5d-bd30-a115c10b5bf1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:21.120514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9f2f3-4c76-4d5d-bd30-a115c10b5bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:21.120874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9f2f3-4c76-4d5d-bd30-a115c10b5bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:21.144565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-32f9f2f3-4c76-4d5d-bd30-a115c10b5bf1 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:21.146817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-32f9f2f3-4c76-4d5d-bd30-a115c10b5bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:21.229458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1266/2577] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:21.240081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d96851-2705-4f28-b003-7209c42fec9a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:21.307306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d96851-2705-4f28-b003-7209c42fec9a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:21.311170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf91224-4dd3-4272-ae86-03101d78403a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:21.311791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf91224-4dd3-4272-ae86-03101d78403a service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:21.312076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf91224-4dd3-4272-ae86-03101d78403a service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:21.312439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf91224-4dd3-4272-ae86-03101d78403a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:21.322445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf91224-4dd3-4272-ae86-03101d78403a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:21.323877 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf91224-4dd3-4272-ae86-03101d78403a service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:21.334717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1312/2578] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:21.379554 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9f2f3-4c76-4d5d-bd30-a115c10b5bf1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:21.381234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-32f9f2f3-4c76-4d5d-bd30-a115c10b5bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:21.394655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9f2f3-4c76-4d5d-bd30-a115c10b5bf1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:21.399758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:21.400429 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:21.400724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:21.401015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] PATH_INFO: `/v3/projects/c2e35bf1f4f74e8d979bae3f767f4757` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:21.403304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e35bf1f4f74e8d979bae3f767f4757)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:21.405434 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:21.573924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e35bf1f4f74e8d979bae3f767f4757', 'request_id': 'req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:21.580127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e35bf1f4f74e8d979bae3f767f4757', 'request_id': 'req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:21.587162 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e35bf1f4f74e8d979bae3f767f4757', 'request_id': 'req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:21.726580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] The token cache is being invalidate because project c2e35bf1f4f74e8d979bae3f767f4757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:21.726908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:21.727777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:21.729640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1267/2579] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:21 2022] DELETE /identity/v3/projects/c2e35bf1f4f74e8d979bae3f767f4757 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:21.733478 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf91224-4dd3-4272-ae86-03101d78403a service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:21.744366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf91224-4dd3-4272-ae86-03101d78403a service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:21.803406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:21.803784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:21.804074 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:21.804358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] PATH_INFO: `/v3/users/e2bfca12470f48fcb7567cc44e01a5b8` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:21.805565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2bfca12470f48fcb7567cc44e01a5b8)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:21.807501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:22.025539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2bfca12470f48fcb7567cc44e01a5b8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:22.043465 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2bfca12470f48fcb7567cc44e01a5b8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:22.045528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2bfca12470f48fcb7567cc44e01a5b8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:22.054943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2bfca12470f48fcb7567cc44e01a5b8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:22.170277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e916f3be-4431-4709-90b8-288587ab496f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2bfca12470f48fcb7567cc44e01a5b8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:22.172453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:22.179782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1313/2580] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:21 2022] DELETE /identity/v3/users/e2bfca12470f48fcb7567cc44e01a5b8 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:22.636604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:22.638378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:22.654790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b358f4-24c5-4b71-bd58-0ffcc7780ddc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:22.701286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2af833-dc0c-4080-a05c-418a4819cdad None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:22.701694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2af833-dc0c-4080-a05c-418a4819cdad None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:22.702048 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2af833-dc0c-4080-a05c-418a4819cdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:22.736763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-de2af833-dc0c-4080-a05c-418a4819cdad None None] MFA Rules not processed for user `caf7d2139ef24629b23f01c08b08a1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:22.739582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-de2af833-dc0c-4080-a05c-418a4819cdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:22.743820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:22.744273 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:22.744659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:22.745044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] PATH_INFO: `/v3/projects/62aa01a5827e4c03b43dfceba5f27e11` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:22.746410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62aa01a5827e4c03b43dfceba5f27e11)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:22.748412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:22.767091 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:22.813055 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1314/2581] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:22 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:22.818126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2af833-dc0c-4080-a05c-418a4819cdad None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:22.819554 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-de2af833-dc0c-4080-a05c-418a4819cdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:22.834345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2af833-dc0c-4080-a05c-418a4819cdad None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:22.839082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dee1f-2ba3-4dd9-8285-c051b8208f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:22.839706 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dee1f-2ba3-4dd9-8285-c051b8208f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:22.840020 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dee1f-2ba3-4dd9-8285-c051b8208f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:22.840327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dee1f-2ba3-4dd9-8285-c051b8208f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:22.841005 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dee1f-2ba3-4dd9-8285-c051b8208f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:22.842966 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dee1f-2ba3-4dd9-8285-c051b8208f8f admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:22.865496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62aa01a5827e4c03b43dfceba5f27e11', 'request_id': 'req-83a4ef94-e1a1-4421-a988-fae830339fd2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:22.870999 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62aa01a5827e4c03b43dfceba5f27e11', 'request_id': 'req-83a4ef94-e1a1-4421-a988-fae830339fd2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:22.877242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62aa01a5827e4c03b43dfceba5f27e11', 'request_id': 'req-83a4ef94-e1a1-4421-a988-fae830339fd2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:22.949243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] The token cache is being invalidate because project 62aa01a5827e4c03b43dfceba5f27e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:22.949504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:22.950127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83a4ef94-e1a1-4421-a988-fae830339fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:22.953750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1315/2582] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:22 2022] POST /identity/v3/projects => generated 405 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:22.953915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1268/2583] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:22 2022] DELETE /identity/v3/projects/62aa01a5827e4c03b43dfceba5f27e11 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:22.954944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dee1f-2ba3-4dd9-8285-c051b8208f8f admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:22.958686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-aa9dee1f-2ba3-4dd9-8285-c051b8208f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:22.958808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8356d1-7414-4471-a9b1-56621a3d320f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:22.958808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8356d1-7414-4471-a9b1-56621a3d320f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:22.958808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8356d1-7414-4471-a9b1-56621a3d320f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:22.978194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-be8356d1-7414-4471-a9b1-56621a3d320f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:22.979471 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-be8356d1-7414-4471-a9b1-56621a3d320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:23.025156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dee1f-2ba3-4dd9-8285-c051b8208f8f admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.029964 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52a357-42a0-4ffc-be70-94e7ec91b917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.030576 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52a357-42a0-4ffc-be70-94e7ec91b917 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.030850 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52a357-42a0-4ffc-be70-94e7ec91b917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.031127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52a357-42a0-4ffc-be70-94e7ec91b917 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.031803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52a357-42a0-4ffc-be70-94e7ec91b917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.033809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52a357-42a0-4ffc-be70-94e7ec91b917 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.042083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1269/2584] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:23.052272 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d74fe-44b7-402e-85c2-d9987980a16a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.052566 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d74fe-44b7-402e-85c2-d9987980a16a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.052850 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d74fe-44b7-402e-85c2-d9987980a16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.074434 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-a49d74fe-44b7-402e-85c2-d9987980a16a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:23.075658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a49d74fe-44b7-402e-85c2-d9987980a16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:23.084427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1270/2585] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:23.088687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d74fe-44b7-402e-85c2-d9987980a16a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.102866 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d74fe-44b7-402e-85c2-d9987980a16a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.103931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671ef76-005c-453c-90de-4d719554d10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.104809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671ef76-005c-453c-90de-4d719554d10c admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.104809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671ef76-005c-453c-90de-4d719554d10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.105130 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671ef76-005c-453c-90de-4d719554d10c admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.105871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671ef76-005c-453c-90de-4d719554d10c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:23.106182 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671ef76-005c-453c-90de-4d719554d10c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.108084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671ef76-005c-453c-90de-4d719554d10c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.121293 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1271/2586] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:36:23.127197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671ef76-005c-453c-90de-4d719554d10c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.138431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671ef76-005c-453c-90de-4d719554d10c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.145232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e418e-7202-4afa-a740-5a66c12150b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.145887 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e418e-7202-4afa-a740-5a66c12150b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.146181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e418e-7202-4afa-a740-5a66c12150b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.146463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e418e-7202-4afa-a740-5a66c12150b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.147136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4e418e-7202-4afa-a740-5a66c12150b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.149107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4e418e-7202-4afa-a740-5a66c12150b6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.153241 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1316/2587] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:22 2022] POST /identity/v3/users => generated 309 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:23.165868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52a357-42a0-4ffc-be70-94e7ec91b917 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.177342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52a357-42a0-4ffc-be70-94e7ec91b917 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.182378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c59658-38c3-45ba-833c-0c21c0f6ba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.183006 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c59658-38c3-45ba-833c-0c21c0f6ba3a admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.183295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c59658-38c3-45ba-833c-0c21c0f6ba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.183580 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c59658-38c3-45ba-833c-0c21c0f6ba3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.184289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c59658-38c3-45ba-833c-0c21c0f6ba3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.185997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c59658-38c3-45ba-833c-0c21c0f6ba3a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.198397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1317/2588] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:23.204174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c59658-38c3-45ba-833c-0c21c0f6ba3a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.212951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c59658-38c3-45ba-833c-0c21c0f6ba3a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.217759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0d6ef-3242-4dd0-b747-3bec2d2505bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.218432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0d6ef-3242-4dd0-b747-3bec2d2505bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.218723 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0d6ef-3242-4dd0-b747-3bec2d2505bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.219010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0d6ef-3242-4dd0-b747-3bec2d2505bf admin admin] PATH_INFO: `/v3/projects/89c773a3dda24ceeb1d476e684eb9a0b/users/ee61289a286b45d5be743ee639865878/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.234717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0d6ef-3242-4dd0-b747-3bec2d2505bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c773a3dda24ceeb1d476e684eb9a0b, user_id=ee61289a286b45d5be743ee639865878, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.238031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0d6ef-3242-4dd0-b747-3bec2d2505bf admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.266118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1272/2589] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:23 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:23.270728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e418e-7202-4afa-a740-5a66c12150b6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.334846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e418e-7202-4afa-a740-5a66c12150b6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.339757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2554bf-d6d8-4c30-8d33-8f05f153ccac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.340400 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2554bf-d6d8-4c30-8d33-8f05f153ccac admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.340725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2554bf-d6d8-4c30-8d33-8f05f153ccac admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.341012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2554bf-d6d8-4c30-8d33-8f05f153ccac admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.341730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2554bf-d6d8-4c30-8d33-8f05f153ccac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.343548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1318/2590] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:23 2022] PUT /identity/v3/projects/89c773a3dda24ceeb1d476e684eb9a0b/users/ee61289a286b45d5be743ee639865878/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:23.343774 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2554bf-d6d8-4c30-8d33-8f05f153ccac admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.352869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0d6ef-3242-4dd0-b747-3bec2d2505bf admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.355205 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:23.377601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:23.419437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0d6ef-3242-4dd0-b747-3bec2d2505bf admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.424562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8671d76-467b-4499-b80c-98494958c7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.425760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671d76-467b-4499-b80c-98494958c7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.426052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671d76-467b-4499-b80c-98494958c7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.426339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671d76-467b-4499-b80c-98494958c7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.427095 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8671d76-467b-4499-b80c-98494958c7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.428711 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8671d76-467b-4499-b80c-98494958c7c7 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.438956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1319/2591] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:23.445802 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8671d76-467b-4499-b80c-98494958c7c7 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.456903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8671d76-467b-4499-b80c-98494958c7c7 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.463484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd546d8-4f28-47b8-aef1-ec5ff38c0147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.464244 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1273/2592] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:23 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:23.464394 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd546d8-4f28-47b8-aef1-ec5ff38c0147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.464624 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd546d8-4f28-47b8-aef1-ec5ff38c0147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.464914 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd546d8-4f28-47b8-aef1-ec5ff38c0147 admin admin] PATH_INFO: `/v3/projects/89c773a3dda24ceeb1d476e684eb9a0b/users/ee61289a286b45d5be743ee639865878/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.467668 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd546d8-4f28-47b8-aef1-ec5ff38c0147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c773a3dda24ceeb1d476e684eb9a0b, user_id=ee61289a286b45d5be743ee639865878, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.468674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2554bf-d6d8-4c30-8d33-8f05f153ccac admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.470509 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd546d8-4f28-47b8-aef1-ec5ff38c0147 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.480677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2554bf-d6d8-4c30-8d33-8f05f153ccac admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.485339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1320/2593] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:23 2022] PUT /identity/v3/projects/89c773a3dda24ceeb1d476e684eb9a0b/users/ee61289a286b45d5be743ee639865878/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:23.542986 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582ff41-6e8b-4bc4-9b76-d836f6f95ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.543622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582ff41-6e8b-4bc4-9b76-d836f6f95ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.543911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582ff41-6e8b-4bc4-9b76-d836f6f95ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.544190 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582ff41-6e8b-4bc4-9b76-d836f6f95ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.544952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582ff41-6e8b-4bc4-9b76-d836f6f95ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.546637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582ff41-6e8b-4bc4-9b76-d836f6f95ee3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.557587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1274/2594] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:23.561814 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd546d8-4f28-47b8-aef1-ec5ff38c0147 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.573775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd546d8-4f28-47b8-aef1-ec5ff38c0147 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.578762 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e72dd-1499-42fc-8268-ccac8c03271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.579388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e72dd-1499-42fc-8268-ccac8c03271b admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.579676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e72dd-1499-42fc-8268-ccac8c03271b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.579967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e72dd-1499-42fc-8268-ccac8c03271b admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.580788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e72dd-1499-42fc-8268-ccac8c03271b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.583298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e72dd-1499-42fc-8268-ccac8c03271b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.594047 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1321/2595] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:23.599813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582ff41-6e8b-4bc4-9b76-d836f6f95ee3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.616357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582ff41-6e8b-4bc4-9b76-d836f6f95ee3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.621433 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985224a-4780-4c1e-9c3c-4ab95fac3b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.622126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985224a-4780-4c1e-9c3c-4ab95fac3b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.622421 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985224a-4780-4c1e-9c3c-4ab95fac3b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.622701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985224a-4780-4c1e-9c3c-4ab95fac3b73 admin admin] PATH_INFO: `/v3/projects/165fb024b2ae4f1a81667043dbed15db/users/dfbfec71add744fbb9ba4662307e1d6a/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.639601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985224a-4780-4c1e-9c3c-4ab95fac3b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=165fb024b2ae4f1a81667043dbed15db, user_id=dfbfec71add744fbb9ba4662307e1d6a, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.642371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985224a-4780-4c1e-9c3c-4ab95fac3b73 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:23.737283 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1275/2596] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:23 2022] PUT /identity/v3/projects/165fb024b2ae4f1a81667043dbed15db/users/dfbfec71add744fbb9ba4662307e1d6a/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:23.747683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a26707-9a60-44b8-aeef-b9c38f07b105 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.747972 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a26707-9a60-44b8-aeef-b9c38f07b105 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.748251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a26707-9a60-44b8-aeef-b9c38f07b105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.781455 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-38a26707-9a60-44b8-aeef-b9c38f07b105 None None] MFA Rules not processed for user `dfbfec71add744fbb9ba4662307e1d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:23.783121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-38a26707-9a60-44b8-aeef-b9c38f07b105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:23.891894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1322/2597] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:23.897450 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985224a-4780-4c1e-9c3c-4ab95fac3b73 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:23.972139 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985224a-4780-4c1e-9c3c-4ab95fac3b73 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:23.978762 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b39e2-4c0c-4072-a9c9-7ac4c247f377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:23.979409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b39e2-4c0c-4072-a9c9-7ac4c247f377 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:23.982738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b39e2-4c0c-4072-a9c9-7ac4c247f377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:23.982738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b39e2-4c0c-4072-a9c9-7ac4c247f377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:23.982738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b39e2-4c0c-4072-a9c9-7ac4c247f377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:23.982738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b39e2-4c0c-4072-a9c9-7ac4c247f377 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:24.071197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1276/2598] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:23 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:24.075521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a26707-9a60-44b8-aeef-b9c38f07b105 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:24.088431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a26707-9a60-44b8-aeef-b9c38f07b105 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:24.167721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5601e-6539-458b-baef-25f147fe4de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:24.168355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5601e-6539-458b-baef-25f147fe4de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:24.168645 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5601e-6539-458b-baef-25f147fe4de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:24.168926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5601e-6539-458b-baef-25f147fe4de8 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:24.169640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5601e-6539-458b-baef-25f147fe4de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:24.171605 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5601e-6539-458b-baef-25f147fe4de8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:24.332215 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1323/2599] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:24 2022] POST /identity/v3/users => generated 294 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:24.336786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b39e2-4c0c-4072-a9c9-7ac4c247f377 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:24.351150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b39e2-4c0c-4072-a9c9-7ac4c247f377 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:24.355552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d487a-2c69-418e-a39f-25351d7f8888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:24.356356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d487a-2c69-418e-a39f-25351d7f8888 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:24.356549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d487a-2c69-418e-a39f-25351d7f8888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:24.361517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d487a-2c69-418e-a39f-25351d7f8888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:24.361517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d487a-2c69-418e-a39f-25351d7f8888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:24.361517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d487a-2c69-418e-a39f-25351d7f8888 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:24.375684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1277/2600] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:24.377334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5601e-6539-458b-baef-25f147fe4de8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:24.388630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5601e-6539-458b-baef-25f147fe4de8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:24.393474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9f056-ced2-405c-9727-e7abbadf8e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:24.394425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9f056-ced2-405c-9727-e7abbadf8e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:24.394881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9f056-ced2-405c-9727-e7abbadf8e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:24.395259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9f056-ced2-405c-9727-e7abbadf8e90 admin admin] PATH_INFO: `/v3/projects/5e262349a02d464e87b450eaaaa0b6ae/users/45c7d3f28429412b997d801647e10578/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:24.404609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:24.412652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9f056-ced2-405c-9727-e7abbadf8e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e262349a02d464e87b450eaaaa0b6ae, user_id=45c7d3f28429412b997d801647e10578, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:24.415402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9f056-ced2-405c-9727-e7abbadf8e90 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:24.560567 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1324/2601] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:24 2022] PUT /identity/v3/projects/5e262349a02d464e87b450eaaaa0b6ae/users/45c7d3f28429412b997d801647e10578/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:24.564479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d487a-2c69-418e-a39f-25351d7f8888 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:24.579665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d487a-2c69-418e-a39f-25351d7f8888 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:24.623239 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:24.647388 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608e9f-fc78-44e9-9e50-5a3132a1d75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:24.648109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608e9f-fc78-44e9-9e50-5a3132a1d75a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:24.648304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608e9f-fc78-44e9-9e50-5a3132a1d75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:24.648582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608e9f-fc78-44e9-9e50-5a3132a1d75a admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:24.649384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3608e9f-fc78-44e9-9e50-5a3132a1d75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:24.654293 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3608e9f-fc78-44e9-9e50-5a3132a1d75a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:24.661386 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1278/2602] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:24.665562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9f056-ced2-405c-9727-e7abbadf8e90 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:24.678816 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9f056-ced2-405c-9727-e7abbadf8e90 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:24.683836 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74615b5d-ce15-4293-bbf8-2fc6e02fa571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:24.685038 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74615b5d-ce15-4293-bbf8-2fc6e02fa571 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:24.685604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74615b5d-ce15-4293-bbf8-2fc6e02fa571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:24.686180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74615b5d-ce15-4293-bbf8-2fc6e02fa571 admin admin] PATH_INFO: `/v3/projects/5e262349a02d464e87b450eaaaa0b6ae/users/45c7d3f28429412b997d801647e10578/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:24.688266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74615b5d-ce15-4293-bbf8-2fc6e02fa571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e262349a02d464e87b450eaaaa0b6ae, user_id=45c7d3f28429412b997d801647e10578, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:24.692067 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74615b5d-ce15-4293-bbf8-2fc6e02fa571 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:24.703105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1325/2603] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:24 2022] PUT /identity/v3/projects/5e262349a02d464e87b450eaaaa0b6ae/users/45c7d3f28429412b997d801647e10578/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:24.715304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858725a-19a5-42df-b17f-feb78507e91b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:24.715593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858725a-19a5-42df-b17f-feb78507e91b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:24.715881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858725a-19a5-42df-b17f-feb78507e91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:24.748318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8858725a-19a5-42df-b17f-feb78507e91b None None] MFA Rules not processed for user `45c7d3f28429412b997d801647e10578`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:24.750294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8858725a-19a5-42df-b17f-feb78507e91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:24.830010 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1279/2604] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:24.842540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74615b5d-ce15-4293-bbf8-2fc6e02fa571 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:24.844318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-74615b5d-ce15-4293-bbf8-2fc6e02fa571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:24.859971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9ec4e-62a9-4d88-abb4-0fdbca31c2fc None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:24.860269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9ec4e-62a9-4d88-abb4-0fdbca31c2fc None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:24.860583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9ec4e-62a9-4d88-abb4-0fdbca31c2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:24.898234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-c6a9ec4e-62a9-4d88-abb4-0fdbca31c2fc None None] MFA Rules not processed for user `ee61289a286b45d5be743ee639865878`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:24.900936 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c6a9ec4e-62a9-4d88-abb4-0fdbca31c2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:24.939979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74615b5d-ce15-4293-bbf8-2fc6e02fa571 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:24.946243 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15352e-37b1-405d-93a9-64834c715fac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:24.946923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15352e-37b1-405d-93a9-64834c715fac service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:24.947212 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15352e-37b1-405d-93a9-64834c715fac service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:24.947536 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15352e-37b1-405d-93a9-64834c715fac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:25.006802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1280/2605] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:25.025790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15352e-37b1-405d-93a9-64834c715fac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:25.027250 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15352e-37b1-405d-93a9-64834c715fac service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:25.037358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1326/2606] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:26.481211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9ec4e-62a9-4d88-abb4-0fdbca31c2fc None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:26.531609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:26.549216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9ec4e-62a9-4d88-abb4-0fdbca31c2fc None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:26.554170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:26.554819 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:26.555108 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:26.555394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] PATH_INFO: `/v3/users/caf7d2139ef24629b23f01c08b08a1a7` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:26.556571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf7d2139ef24629b23f01c08b08a1a7)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:26.558583 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:26.746771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf7d2139ef24629b23f01c08b08a1a7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:26.759363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf7d2139ef24629b23f01c08b08a1a7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:26.766270 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf7d2139ef24629b23f01c08b08a1a7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:26.840261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf7d2139ef24629b23f01c08b08a1a7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:26.841252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf7d2139ef24629b23f01c08b08a1a7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:26.851548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1281/2607] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:26 2022] DELETE /identity/v3/users/caf7d2139ef24629b23f01c08b08a1a7 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:27.107253 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15352e-37b1-405d-93a9-64834c715fac service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:27.120425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15352e-37b1-405d-93a9-64834c715fac service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:27.195172 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:27.195874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:27.196167 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:27.196477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] PATH_INFO: `/v3/projects/363aa7c3840443ae8d314eb69d382045` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:27.197714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=363aa7c3840443ae8d314eb69d382045)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:27.199683 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:27.296126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:27.309750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '363aa7c3840443ae8d314eb69d382045', 'request_id': 'req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:27.315237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '363aa7c3840443ae8d314eb69d382045', 'request_id': 'req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:27.420379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '363aa7c3840443ae8d314eb69d382045', 'request_id': 'req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:27.433287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] The token cache is being invalidate because project 363aa7c3840443ae8d314eb69d382045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:27.433611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:27.434289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:27.435819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1327/2608] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:27 2022] DELETE /identity/v3/projects/363aa7c3840443ae8d314eb69d382045 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:27.441415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:27.443435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:27.459040 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8665a3f-f2df-498b-8fbf-eea4022978b9 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:27.521698 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:27.522394 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:27.522627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:27.522911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] PATH_INFO: `/v3/users/ee61289a286b45d5be743ee639865878` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:27.524169 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee61289a286b45d5be743ee639865878)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:27.526247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:27.796793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee61289a286b45d5be743ee639865878'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:27.809620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee61289a286b45d5be743ee639865878'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:27.810859 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:27.818118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee61289a286b45d5be743ee639865878'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:27.906346 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee61289a286b45d5be743ee639865878'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:27.906985 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee61289a286b45d5be743ee639865878'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:27.914029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1282/2609] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:27 2022] DELETE /identity/v3/users/ee61289a286b45d5be743ee639865878 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:28.154530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:28.155944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:28.179695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b0b7d-e5e7-4bbe-98c5-4f16be347a70 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:28.246861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:28.247524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:28.248297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:28.248297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] PATH_INFO: `/v3/projects/89c773a3dda24ceeb1d476e684eb9a0b` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:28.249259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] RBAC: Authorizing `identity:delete_project(project_id=89c773a3dda24ceeb1d476e684eb9a0b)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:28.251213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:28.268978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:28.270589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:28.287030 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806d5da-7a6c-41d8-b803-c15597474b33 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:28.291924 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:28.292575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:28.292862 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:28.293143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] PATH_INFO: `/v3/users/dfbfec71add744fbb9ba4662307e1d6a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:28.294428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfbfec71add744fbb9ba4662307e1d6a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:28.296371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:28.385818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c773a3dda24ceeb1d476e684eb9a0b', 'request_id': 'req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:28.391609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c773a3dda24ceeb1d476e684eb9a0b', 'request_id': 'req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:28.502006 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c773a3dda24ceeb1d476e684eb9a0b', 'request_id': 'req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:28.519219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] The token cache is being invalidate because project 89c773a3dda24ceeb1d476e684eb9a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:28.519542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:28.520195 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c9e9032b-9b9d-41e6-9d7d-b0b4d724769d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:28.521767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1328/2610] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:28 2022] DELETE /identity/v3/projects/89c773a3dda24ceeb1d476e684eb9a0b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:28.526436 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92845ff7-4c70-40ed-87f9-4156ef0a46cc None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:28.532213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92845ff7-4c70-40ed-87f9-4156ef0a46cc None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:28.532213 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92845ff7-4c70-40ed-87f9-4156ef0a46cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:28.540802 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfbfec71add744fbb9ba4662307e1d6a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:28.548402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-92845ff7-4c70-40ed-87f9-4156ef0a46cc None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:28.549688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-92845ff7-4c70-40ed-87f9-4156ef0a46cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:28.554107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfbfec71add744fbb9ba4662307e1d6a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:28.561530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfbfec71add744fbb9ba4662307e1d6a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:28.595774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:28.612351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1329/2611] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:28.620633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338dc3c-70e3-497f-939a-ffb1a8475171 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:28.620847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338dc3c-70e3-497f-939a-ffb1a8475171 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:28.621133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338dc3c-70e3-497f-939a-ffb1a8475171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:28.640375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4338dc3c-70e3-497f-939a-ffb1a8475171 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:28.641712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4338dc3c-70e3-497f-939a-ffb1a8475171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:28.643302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfbfec71add744fbb9ba4662307e1d6a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:28.643941 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-57604d19-b70d-44a2-9a28-84b579c2c719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfbfec71add744fbb9ba4662307e1d6a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:28.649837 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1330/2612] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:28.650771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1283/2613] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:28 2022] DELETE /identity/v3/users/dfbfec71add744fbb9ba4662307e1d6a => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:28.654085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338dc3c-70e3-497f-939a-ffb1a8475171 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:28.654879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d51c3-0713-4894-b54f-74417ad2a634 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:28.655180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d51c3-0713-4894-b54f-74417ad2a634 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:28.655469 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d51c3-0713-4894-b54f-74417ad2a634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:28.664525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338dc3c-70e3-497f-939a-ffb1a8475171 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:28.675413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-d55d51c3-0713-4894-b54f-74417ad2a634 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:28.676714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-d55d51c3-0713-4894-b54f-74417ad2a634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:28.726133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128b20f-c814-4eb8-81f2-94995e07f75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:28.726771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128b20f-c814-4eb8-81f2-94995e07f75d admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:28.727059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128b20f-c814-4eb8-81f2-94995e07f75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:28.727343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128b20f-c814-4eb8-81f2-94995e07f75d admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:28.728052 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128b20f-c814-4eb8-81f2-94995e07f75d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:28.728364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128b20f-c814-4eb8-81f2-94995e07f75d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:28.730051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128b20f-c814-4eb8-81f2-94995e07f75d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:28.741625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1284/2614] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:28.746430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1331/2615] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:36:28.747481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d51c3-0713-4894-b54f-74417ad2a634 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:28.751403 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128b20f-c814-4eb8-81f2-94995e07f75d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:28.754090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6128b20f-c814-4eb8-81f2-94995e07f75d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:28.757367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d51c3-0713-4894-b54f-74417ad2a634 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:28.762520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f42b3b-d876-4b6b-bd19-a8c82e8d152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:28.763143 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f42b3b-d876-4b6b-bd19-a8c82e8d152f admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:28.763425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f42b3b-d876-4b6b-bd19-a8c82e8d152f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:28.763700 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f42b3b-d876-4b6b-bd19-a8c82e8d152f admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:28.764368 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f42b3b-d876-4b6b-bd19-a8c82e8d152f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:28.766324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f42b3b-d876-4b6b-bd19-a8c82e8d152f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:28.836273 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128b20f-c814-4eb8-81f2-94995e07f75d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:28.840076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21e53-6b02-4f2f-af30-591959245ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:28.840702 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21e53-6b02-4f2f-af30-591959245ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:28.840991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21e53-6b02-4f2f-af30-591959245ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:28.841279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21e53-6b02-4f2f-af30-591959245ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:28.845018 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1285/2616] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:28 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:28.849452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f42b3b-d876-4b6b-bd19-a8c82e8d152f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:28.851648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e21e53-6b02-4f2f-af30-591959245ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:28.853089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e21e53-6b02-4f2f-af30-591959245ae7 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:28.863422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1332/2617] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:36:28.886601 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:28.914538 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f42b3b-d876-4b6b-bd19-a8c82e8d152f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:28.924267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72849f-a5c7-4018-9690-93565542c606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:28.924857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72849f-a5c7-4018-9690-93565542c606 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:28.925148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72849f-a5c7-4018-9690-93565542c606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:28.925491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72849f-a5c7-4018-9690-93565542c606 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:28.926206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72849f-a5c7-4018-9690-93565542c606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:28.929179 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72849f-a5c7-4018-9690-93565542c606 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:29.084009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1286/2618] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:28 2022] POST /identity/v3/users => generated 291 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:29.088380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21e53-6b02-4f2f-af30-591959245ae7 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:29.098564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21e53-6b02-4f2f-af30-591959245ae7 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:29.104062 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501d040-bf4b-4f2e-ad30-e302cebc2367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:29.104733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501d040-bf4b-4f2e-ad30-e302cebc2367 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:29.105753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501d040-bf4b-4f2e-ad30-e302cebc2367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:29.106271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501d040-bf4b-4f2e-ad30-e302cebc2367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:29.107594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501d040-bf4b-4f2e-ad30-e302cebc2367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:29.109496 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501d040-bf4b-4f2e-ad30-e302cebc2367 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:29.124859 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1333/2619] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:29.127886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72849f-a5c7-4018-9690-93565542c606 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:29.139079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72849f-a5c7-4018-9690-93565542c606 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:29.144194 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d10526-1de7-44dd-ba65-617709a81f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:29.144823 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d10526-1de7-44dd-ba65-617709a81f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:29.145107 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d10526-1de7-44dd-ba65-617709a81f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:29.145439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d10526-1de7-44dd-ba65-617709a81f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:29.146223 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d10526-1de7-44dd-ba65-617709a81f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:29.148078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d10526-1de7-44dd-ba65-617709a81f06 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:29.159779 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1287/2620] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:29.164982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501d040-bf4b-4f2e-ad30-e302cebc2367 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:29.175482 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501d040-bf4b-4f2e-ad30-e302cebc2367 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:29.181143 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b3412-9b0b-4e35-9c1d-8f9d5efd9e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:29.181867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b3412-9b0b-4e35-9c1d-8f9d5efd9e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:29.182239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b3412-9b0b-4e35-9c1d-8f9d5efd9e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:29.182524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b3412-9b0b-4e35-9c1d-8f9d5efd9e97 admin admin] PATH_INFO: `/v3/projects/8ac6674505224cb58374b5976f14167b/users/0c7d93ae05024369ab4c30270d9f0ac4/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:29.200063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b3412-9b0b-4e35-9c1d-8f9d5efd9e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac6674505224cb58374b5976f14167b, user_id=0c7d93ae05024369ab4c30270d9f0ac4, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:29.202766 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b3412-9b0b-4e35-9c1d-8f9d5efd9e97 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:29.280220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1334/2621] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:29 2022] PUT /identity/v3/projects/8ac6674505224cb58374b5976f14167b/users/0c7d93ae05024369ab4c30270d9f0ac4/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:29.293456 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0b6e1-6ad9-40ed-9a28-f66bce194fa8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:29.293752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0b6e1-6ad9-40ed-9a28-f66bce194fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:29.294039 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0b6e1-6ad9-40ed-9a28-f66bce194fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:29.323165 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-17e0b6e1-6ad9-40ed-9a28-f66bce194fa8 None None] MFA Rules not processed for user `0c7d93ae05024369ab4c30270d9f0ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:29.324375 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-17e0b6e1-6ad9-40ed-9a28-f66bce194fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:29.351461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b3412-9b0b-4e35-9c1d-8f9d5efd9e97 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:29.353378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-095b3412-9b0b-4e35-9c1d-8f9d5efd9e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:29.404378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1288/2622] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:29.408618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0b6e1-6ad9-40ed-9a28-f66bce194fa8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:29.408770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:29.450083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b3412-9b0b-4e35-9c1d-8f9d5efd9e97 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:29.455996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:29.456738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:29.457070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:29.458259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] PATH_INFO: `/v3/projects/165fb024b2ae4f1a81667043dbed15db` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:29.460204 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=165fb024b2ae4f1a81667043dbed15db)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:29.462281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:29.485964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0b6e1-6ad9-40ed-9a28-f66bce194fa8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:29.491874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f68f-222d-40f1-b185-8eea583d1c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:29.492520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f68f-222d-40f1-b185-8eea583d1c37 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:29.492808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f68f-222d-40f1-b185-8eea583d1c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:29.493090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f68f-222d-40f1-b185-8eea583d1c37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:29.493798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0f68f-222d-40f1-b185-8eea583d1c37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:29.495774 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0f68f-222d-40f1-b185-8eea583d1c37 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:29.584904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1289/2623] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:29 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:29.588400 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '165fb024b2ae4f1a81667043dbed15db', 'request_id': 'req-1fd92f98-e6e1-47fe-a62c-f6431e107b35'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:29.588735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f68f-222d-40f1-b185-8eea583d1c37 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:29.595013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '165fb024b2ae4f1a81667043dbed15db', 'request_id': 'req-1fd92f98-e6e1-47fe-a62c-f6431e107b35'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:29.633093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:29.645226 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '165fb024b2ae4f1a81667043dbed15db', 'request_id': 'req-1fd92f98-e6e1-47fe-a62c-f6431e107b35'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:29.657487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f68f-222d-40f1-b185-8eea583d1c37 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:29.662697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] The token cache is being invalidate because project 165fb024b2ae4f1a81667043dbed15db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:29.662991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:29.663679 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:29.668259 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1335/2624] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:29 2022] DELETE /identity/v3/projects/165fb024b2ae4f1a81667043dbed15db => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:29.668417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66902a-20b1-4421-bd13-3878ed66d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:29.668417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66902a-20b1-4421-bd13-3878ed66d033 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:29.668417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66902a-20b1-4421-bd13-3878ed66d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:29.668417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66902a-20b1-4421-bd13-3878ed66d033 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:29.669218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66902a-20b1-4421-bd13-3878ed66d033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:29.669852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:29.671320 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:29.671842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66902a-20b1-4421-bd13-3878ed66d033 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:29.743692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd92f98-e6e1-47fe-a62c-f6431e107b35 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:29.748858 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:29.749541 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:29.749857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:29.750176 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] PATH_INFO: `/v3/users/45c7d3f28429412b997d801647e10578` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:29.751442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c7d3f28429412b997d801647e10578)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:29.753365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:29.768904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1290/2625] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:29 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:29.774588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66902a-20b1-4421-bd13-3878ed66d033 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:29.776345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7d66902a-20b1-4421-bd13-3878ed66d033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:29.790061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66902a-20b1-4421-bd13-3878ed66d033 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:29.795300 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f9937-b0ec-4f27-8a78-5250674bac2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:29.796128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f9937-b0ec-4f27-8a78-5250674bac2c admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:29.796607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f9937-b0ec-4f27-8a78-5250674bac2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:29.796965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f9937-b0ec-4f27-8a78-5250674bac2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:29.797817 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f9937-b0ec-4f27-8a78-5250674bac2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:29.799428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f9937-b0ec-4f27-8a78-5250674bac2c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:29.813350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1291/2626] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:29.814339 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f9937-b0ec-4f27-8a78-5250674bac2c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:29.824550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f9937-b0ec-4f27-8a78-5250674bac2c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:29.829674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f213d96-f5d9-43ab-8bc5-b62670c0596c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:29.830384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f213d96-f5d9-43ab-8bc5-b62670c0596c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:29.830760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f213d96-f5d9-43ab-8bc5-b62670c0596c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:29.831123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f213d96-f5d9-43ab-8bc5-b62670c0596c admin admin] PATH_INFO: `/v3/projects/d68baa42375b44038328b82c8243c069/users/9694f4f0bd6848c2906acb1f0009e221/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:29.850371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f213d96-f5d9-43ab-8bc5-b62670c0596c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d68baa42375b44038328b82c8243c069, user_id=9694f4f0bd6848c2906acb1f0009e221, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:29.853052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f213d96-f5d9-43ab-8bc5-b62670c0596c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:29.943865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c7d3f28429412b997d801647e10578'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:29.951010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:29.956352 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c7d3f28429412b997d801647e10578'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:29.957068 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c7d3f28429412b997d801647e10578'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:29.964314 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1292/2627] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:29 2022] PUT /identity/v3/projects/d68baa42375b44038328b82c8243c069/users/9694f4f0bd6848c2906acb1f0009e221/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:29.965121 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c7d3f28429412b997d801647e10578'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:29.968803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f213d96-f5d9-43ab-8bc5-b62670c0596c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:30.031965 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f213d96-f5d9-43ab-8bc5-b62670c0596c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:30.037508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856e8dd-6fa6-41e5-9c9b-430ef7e06068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:30.038161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856e8dd-6fa6-41e5-9c9b-430ef7e06068 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:30.038472 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856e8dd-6fa6-41e5-9c9b-430ef7e06068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:30.038747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856e8dd-6fa6-41e5-9c9b-430ef7e06068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:30.039693 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856e8dd-6fa6-41e5-9c9b-430ef7e06068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:30.041565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856e8dd-6fa6-41e5-9c9b-430ef7e06068 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:30.054491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1293/2628] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:30.058749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856e8dd-6fa6-41e5-9c9b-430ef7e06068 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:30.068972 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856e8dd-6fa6-41e5-9c9b-430ef7e06068 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:30.073701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a9897-2791-4cc9-a3b7-b4aeabf9f451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:30.074341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a9897-2791-4cc9-a3b7-b4aeabf9f451 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:30.074626 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a9897-2791-4cc9-a3b7-b4aeabf9f451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:30.074974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a9897-2791-4cc9-a3b7-b4aeabf9f451 admin admin] PATH_INFO: `/v3/projects/d68baa42375b44038328b82c8243c069/users/9694f4f0bd6848c2906acb1f0009e221/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:30.077542 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a9897-2791-4cc9-a3b7-b4aeabf9f451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d68baa42375b44038328b82c8243c069, user_id=9694f4f0bd6848c2906acb1f0009e221, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:30.080516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a9897-2791-4cc9-a3b7-b4aeabf9f451 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:30.091565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1294/2629] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:30 2022] PUT /identity/v3/projects/d68baa42375b44038328b82c8243c069/users/9694f4f0bd6848c2906acb1f0009e221/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:30.111300 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486a0a7-0ca8-4bfe-98d5-5e83ff12dc1e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:30.111594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486a0a7-0ca8-4bfe-98d5-5e83ff12dc1e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:30.111877 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486a0a7-0ca8-4bfe-98d5-5e83ff12dc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:30.142148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8486a0a7-0ca8-4bfe-98d5-5e83ff12dc1e None None] MFA Rules not processed for user `9694f4f0bd6848c2906acb1f0009e221`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:30.143416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8486a0a7-0ca8-4bfe-98d5-5e83ff12dc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:30.188378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c7d3f28429412b997d801647e10578'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:30.195466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:30.195643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1336/2630] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:29 2022] DELETE /identity/v3/users/45c7d3f28429412b997d801647e10578 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:30.225051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1295/2631] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:30.238404 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:30.240660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:30.313907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c12d6-c052-4533-ba73-507eef12891e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:30.317715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003d3ea-3efa-4f0f-95fa-2fd6953be3fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:30.318423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d3ea-3efa-4f0f-95fa-2fd6953be3fa service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:30.318744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d3ea-3efa-4f0f-95fa-2fd6953be3fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:30.319104 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d3ea-3efa-4f0f-95fa-2fd6953be3fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:30.382817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003d3ea-3efa-4f0f-95fa-2fd6953be3fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:30.384494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003d3ea-3efa-4f0f-95fa-2fd6953be3fa service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:30.394377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1337/2632] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:30.687127 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486a0a7-0ca8-4bfe-98d5-5e83ff12dc1e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:30.689565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8486a0a7-0ca8-4bfe-98d5-5e83ff12dc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:30.708166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003d3ea-3efa-4f0f-95fa-2fd6953be3fa service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:30.729265 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003d3ea-3efa-4f0f-95fa-2fd6953be3fa service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:30.733471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c167a0-0c9a-4709-8b0d-0eff755f85e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:30.734099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c167a0-0c9a-4709-8b0d-0eff755f85e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:30.734388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c167a0-0c9a-4709-8b0d-0eff755f85e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:30.734790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c167a0-0c9a-4709-8b0d-0eff755f85e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:30.736529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-03c167a0-0c9a-4709-8b0d-0eff755f85e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:30.769296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486a0a7-0ca8-4bfe-98d5-5e83ff12dc1e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:30.800203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c167a0-0c9a-4709-8b0d-0eff755f85e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:30.806767 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c167a0-0c9a-4709-8b0d-0eff755f85e0 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:30.811733 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1338/2633] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:36:30.829484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:30.830154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:30.830439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:30.830718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] PATH_INFO: `/v3/projects/5e262349a02d464e87b450eaaaa0b6ae` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:30.831948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e262349a02d464e87b450eaaaa0b6ae)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:30.835268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:30.953753 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e262349a02d464e87b450eaaaa0b6ae', 'request_id': 'req-d640f790-a34d-479c-9acd-202ab1c912b7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:30.959466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e262349a02d464e87b450eaaaa0b6ae', 'request_id': 'req-d640f790-a34d-479c-9acd-202ab1c912b7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:30.965882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e262349a02d464e87b450eaaaa0b6ae', 'request_id': 'req-d640f790-a34d-479c-9acd-202ab1c912b7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:31.067769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] The token cache is being invalidate because project 5e262349a02d464e87b450eaaaa0b6ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:31.068073 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:31.068837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-d640f790-a34d-479c-9acd-202ab1c912b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:31.070449 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1296/2634] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:30 2022] DELETE /identity/v3/projects/5e262349a02d464e87b450eaaaa0b6ae => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:31.075525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562a301-7b5b-4f08-a12b-eb9804cd3bcf None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:31.075822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562a301-7b5b-4f08-a12b-eb9804cd3bcf None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:31.076107 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562a301-7b5b-4f08-a12b-eb9804cd3bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:31.096399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-7562a301-7b5b-4f08-a12b-eb9804cd3bcf None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:31.097803 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7562a301-7b5b-4f08-a12b-eb9804cd3bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:31.107701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:31.158923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1339/2635] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:31.167536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f419b4e-d0cb-4c83-9b8a-a370dbbed05f None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:31.167827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f419b4e-d0cb-4c83-9b8a-a370dbbed05f None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:31.168212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f419b4e-d0cb-4c83-9b8a-a370dbbed05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:31.188632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7f419b4e-d0cb-4c83-9b8a-a370dbbed05f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:31.189873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7f419b4e-d0cb-4c83-9b8a-a370dbbed05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:31.198695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1297/2636] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:31.203161 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562a301-7b5b-4f08-a12b-eb9804cd3bcf None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:31.217428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562a301-7b5b-4f08-a12b-eb9804cd3bcf None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:31.225672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da506864-974d-4612-83a2-a61d979f2e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:31.226319 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da506864-974d-4612-83a2-a61d979f2e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:31.226613 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da506864-974d-4612-83a2-a61d979f2e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:31.226956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da506864-974d-4612-83a2-a61d979f2e7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:31.227763 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da506864-974d-4612-83a2-a61d979f2e7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:31.228160 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da506864-974d-4612-83a2-a61d979f2e7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:31.229894 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da506864-974d-4612-83a2-a61d979f2e7d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:31.244261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1340/2637] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:36:31.248925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f419b4e-d0cb-4c83-9b8a-a370dbbed05f None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:31.258992 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f419b4e-d0cb-4c83-9b8a-a370dbbed05f None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:31.264137 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da4845-e108-4e89-a63c-7d67bf1ca49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:31.267629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da4845-e108-4e89-a63c-7d67bf1ca49e admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:31.267629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da4845-e108-4e89-a63c-7d67bf1ca49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:31.267629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da4845-e108-4e89-a63c-7d67bf1ca49e admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:31.267629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da4845-e108-4e89-a63c-7d67bf1ca49e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:31.268264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da4845-e108-4e89-a63c-7d67bf1ca49e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:31.458054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:31.469208 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1298/2638] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:31 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:31.470388 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da506864-974d-4612-83a2-a61d979f2e7d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:31.481337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da506864-974d-4612-83a2-a61d979f2e7d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:31.539282 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e017d-4960-41e7-838e-9ba933830fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:31.540010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e017d-4960-41e7-838e-9ba933830fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:31.540433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e017d-4960-41e7-838e-9ba933830fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:31.540806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e017d-4960-41e7-838e-9ba933830fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:31.541594 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e017d-4960-41e7-838e-9ba933830fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:31.545815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e017d-4960-41e7-838e-9ba933830fd8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:31.633504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1341/2639] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:31 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:31.637594 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da4845-e108-4e89-a63c-7d67bf1ca49e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:31.648645 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da4845-e108-4e89-a63c-7d67bf1ca49e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:31.653511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9d298-6cac-4bd4-802d-5064d1ae3c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:31.654080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d298-6cac-4bd4-802d-5064d1ae3c88 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:31.654367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d298-6cac-4bd4-802d-5064d1ae3c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:31.654758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d298-6cac-4bd4-802d-5064d1ae3c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:31.655426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9d298-6cac-4bd4-802d-5064d1ae3c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:31.657049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9d298-6cac-4bd4-802d-5064d1ae3c88 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:31.670533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1299/2640] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:31.671741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e017d-4960-41e7-838e-9ba933830fd8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:31.682891 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e017d-4960-41e7-838e-9ba933830fd8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:31.687695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbea170-1b81-4bbd-aa18-f291fd175348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:31.697378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbea170-1b81-4bbd-aa18-f291fd175348 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:31.697378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbea170-1b81-4bbd-aa18-f291fd175348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:31.697378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbea170-1b81-4bbd-aa18-f291fd175348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:31.697378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbea170-1b81-4bbd-aa18-f291fd175348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:31.697378 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbea170-1b81-4bbd-aa18-f291fd175348 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:31.703260 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1342/2641] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:31.708525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9d298-6cac-4bd4-802d-5064d1ae3c88 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:31.719892 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9d298-6cac-4bd4-802d-5064d1ae3c88 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:31.727528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20fbbb-6b98-4cc7-a6c7-903b35b3ff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:31.727528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20fbbb-6b98-4cc7-a6c7-903b35b3ff7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:31.727528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20fbbb-6b98-4cc7-a6c7-903b35b3ff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:31.727528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20fbbb-6b98-4cc7-a6c7-903b35b3ff7f admin admin] PATH_INFO: `/v3/projects/3f8d30bf32cf4aab8ca1cfbce133de80/users/ba82cc46e4fe4d2c8a39a6d23e03145e/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:31.742264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20fbbb-6b98-4cc7-a6c7-903b35b3ff7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f8d30bf32cf4aab8ca1cfbce133de80, user_id=ba82cc46e4fe4d2c8a39a6d23e03145e, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:31.745080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20fbbb-6b98-4cc7-a6c7-903b35b3ff7f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:31.835006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1300/2642] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:31 2022] PUT /identity/v3/projects/3f8d30bf32cf4aab8ca1cfbce133de80/users/ba82cc46e4fe4d2c8a39a6d23e03145e/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:31.845345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ff9ea-f0e9-4600-ac71-95a1877830ae None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:31.845652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ff9ea-f0e9-4600-ac71-95a1877830ae None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:31.845954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ff9ea-f0e9-4600-ac71-95a1877830ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:31.875657 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-050ff9ea-f0e9-4600-ac71-95a1877830ae None None] MFA Rules not processed for user `ba82cc46e4fe4d2c8a39a6d23e03145e`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:31.876890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-050ff9ea-f0e9-4600-ac71-95a1877830ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:31.942003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1343/2643] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:31.953881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20fbbb-6b98-4cc7-a6c7-903b35b3ff7f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:31.955711 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3a20fbbb-6b98-4cc7-a6c7-903b35b3ff7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:32.026716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20fbbb-6b98-4cc7-a6c7-903b35b3ff7f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:32.030448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec193b68-9adc-47e0-b246-8d829af60ff3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:32.031075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec193b68-9adc-47e0-b246-8d829af60ff3 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:32.031363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec193b68-9adc-47e0-b246-8d829af60ff3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:32.031644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec193b68-9adc-47e0-b246-8d829af60ff3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:32.040881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec193b68-9adc-47e0-b246-8d829af60ff3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:32.042383 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec193b68-9adc-47e0-b246-8d829af60ff3 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:32.051199 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1301/2644] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:33.242939 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ff9ea-f0e9-4600-ac71-95a1877830ae None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:33.244448 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-050ff9ea-f0e9-4600-ac71-95a1877830ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:33.254996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:33.311408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ff9ea-f0e9-4600-ac71-95a1877830ae None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:33.316297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:33.317010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:33.317323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:33.317642 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] PATH_INFO: `/v3/users/54e5615d17504e25836a2d4c43dc4022` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:33.319530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e5615d17504e25836a2d4c43dc4022)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:33.321506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:33.510483 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e5615d17504e25836a2d4c43dc4022'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:33.522772 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e5615d17504e25836a2d4c43dc4022'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:33.523528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e5615d17504e25836a2d4c43dc4022'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:33.530565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e5615d17504e25836a2d4c43dc4022'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:33.597285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e5615d17504e25836a2d4c43dc4022'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:33.603839 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1344/2645] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:33 2022] DELETE /identity/v3/users/54e5615d17504e25836a2d4c43dc4022 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:33.608362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b8e1-680e-4d2b-8ef3-58cde6281d93 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:33.608899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b8e1-680e-4d2b-8ef3-58cde6281d93 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:33.608899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b8e1-680e-4d2b-8ef3-58cde6281d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:33.629591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8826b8e1-680e-4d2b-8ef3-58cde6281d93 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:33.629902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8826b8e1-680e-4d2b-8ef3-58cde6281d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:33.693603 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1302/2646] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:33.704784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:33.706235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:33.766387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b4d9e-3b93-48b1-8bd9-59b360d1276d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:33.770119 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682b0e3-2d29-4b25-9115-9765c386cc3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:33.770746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682b0e3-2d29-4b25-9115-9765c386cc3f service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:33.771034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682b0e3-2d29-4b25-9115-9765c386cc3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:33.771314 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682b0e3-2d29-4b25-9115-9765c386cc3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:33.781423 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682b0e3-2d29-4b25-9115-9765c386cc3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:33.782916 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682b0e3-2d29-4b25-9115-9765c386cc3f service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:33.797375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1345/2647] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:34.210454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826b8e1-680e-4d2b-8ef3-58cde6281d93 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:34.212312 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8826b8e1-680e-4d2b-8ef3-58cde6281d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:34.225463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826b8e1-680e-4d2b-8ef3-58cde6281d93 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:34.235927 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:34.236771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:34.237090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:34.237478 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] PATH_INFO: `/v3/projects/19010298549c4be5b1946378e8c719eb` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:34.238666 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19010298549c4be5b1946378e8c719eb)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:34.240617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:34.513064 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19010298549c4be5b1946378e8c719eb', 'request_id': 'req-c8339d1b-ab4e-4877-9e05-644b431b3b97'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:34.514336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:34.519157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19010298549c4be5b1946378e8c719eb', 'request_id': 'req-c8339d1b-ab4e-4877-9e05-644b431b3b97'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:34.525683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19010298549c4be5b1946378e8c719eb', 'request_id': 'req-c8339d1b-ab4e-4877-9e05-644b431b3b97'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:34.650409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] The token cache is being invalidate because project 19010298549c4be5b1946378e8c719eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:34.651181 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:34.651305 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:34.652839 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1303/2648] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:34 2022] DELETE /identity/v3/projects/19010298549c4be5b1946378e8c719eb => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:34.656905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682b0e3-2d29-4b25-9115-9765c386cc3f service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:34.667716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682b0e3-2d29-4b25-9115-9765c386cc3f service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:34.697950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:34.725990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:34.726822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:34.727111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:34.727389 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] PATH_INFO: `/v3/users/1b0a928d9cfb4c318b7e18ad6247c2d3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:34.728638 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b0a928d9cfb4c318b7e18ad6247c2d3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:34.730612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:34.915255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b0a928d9cfb4c318b7e18ad6247c2d3'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:34.927666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b0a928d9cfb4c318b7e18ad6247c2d3'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:34.928527 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b0a928d9cfb4c318b7e18ad6247c2d3'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:34.936042 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b0a928d9cfb4c318b7e18ad6247c2d3'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:35.003363 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d98d7944-d53e-4b93-90ba-b8ef74d185df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b0a928d9cfb4c318b7e18ad6247c2d3'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:35.010341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1346/2649] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:34 2022] DELETE /identity/v3/users/1b0a928d9cfb4c318b7e18ad6247c2d3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:35.446973 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:35.448514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:35.462528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339d1b-ab4e-4877-9e05-644b431b3b97 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:35.519585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:35.520234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:35.520521 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:35.520835 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] PATH_INFO: `/v3/projects/bd9bc28b775e4909937d42509b27bf93` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:35.522216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd9bc28b775e4909937d42509b27bf93)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:35.524279 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:35.668509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd9bc28b775e4909937d42509b27bf93', 'request_id': 'req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:35.674277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd9bc28b775e4909937d42509b27bf93', 'request_id': 'req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:35.681512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd9bc28b775e4909937d42509b27bf93', 'request_id': 'req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:35.759055 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] The token cache is being invalidate because project bd9bc28b775e4909937d42509b27bf93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:35.759422 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:35.759982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-8aef32d5-f8eb-45ad-9dbe-26a25a133d44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:35.761536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1304/2650] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:35 2022] DELETE /identity/v3/projects/bd9bc28b775e4909937d42509b27bf93 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:35.766524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaef91-27df-4368-ab64-5ea6ebe324d4 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:35.767003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaef91-27df-4368-ab64-5ea6ebe324d4 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:35.767122 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaef91-27df-4368-ab64-5ea6ebe324d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:35.787305 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-31aaef91-27df-4368-ab64-5ea6ebe324d4 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:35.789214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-31aaef91-27df-4368-ab64-5ea6ebe324d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:35.851884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1347/2651] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:35.859901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e8b6b-e5c7-43bd-a52e-b113905ee55a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:35.860183 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e8b6b-e5c7-43bd-a52e-b113905ee55a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:35.860457 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e8b6b-e5c7-43bd-a52e-b113905ee55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:35.880507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-bb8e8b6b-e5c7-43bd-a52e-b113905ee55a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:35.881844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-bb8e8b6b-e5c7-43bd-a52e-b113905ee55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:35.890420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1305/2652] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:35.894555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaef91-27df-4368-ab64-5ea6ebe324d4 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:35.905271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaef91-27df-4368-ab64-5ea6ebe324d4 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:35.915342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124afcb-3d5f-4ede-895e-60bf40c976c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:35.915979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124afcb-3d5f-4ede-895e-60bf40c976c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:35.916271 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124afcb-3d5f-4ede-895e-60bf40c976c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:35.916617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124afcb-3d5f-4ede-895e-60bf40c976c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:35.917364 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124afcb-3d5f-4ede-895e-60bf40c976c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:35.917678 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124afcb-3d5f-4ede-895e-60bf40c976c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:35.919318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124afcb-3d5f-4ede-895e-60bf40c976c4 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:35.931431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1348/2653] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:36:35.935799 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e8b6b-e5c7-43bd-a52e-b113905ee55a None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:35.945353 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e8b6b-e5c7-43bd-a52e-b113905ee55a None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:35.950124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06d70c-6239-4fab-9b04-078e387771a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:35.950742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06d70c-6239-4fab-9b04-078e387771a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:35.951028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06d70c-6239-4fab-9b04-078e387771a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:35.951307 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06d70c-6239-4fab-9b04-078e387771a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:35.951988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06d70c-6239-4fab-9b04-078e387771a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:35.953939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06d70c-6239-4fab-9b04-078e387771a4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:36.107807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1306/2654] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:35 2022] POST /identity/v3/projects => generated 365 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:36.112009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124afcb-3d5f-4ede-895e-60bf40c976c4 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:36.122501 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124afcb-3d5f-4ede-895e-60bf40c976c4 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:36.158868 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:36.183636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a62b81-8e61-469c-ac6f-dc64b03a07c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:36.184584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a62b81-8e61-469c-ac6f-dc64b03a07c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:36.185016 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a62b81-8e61-469c-ac6f-dc64b03a07c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:36.185484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a62b81-8e61-469c-ac6f-dc64b03a07c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:36.186516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a62b81-8e61-469c-ac6f-dc64b03a07c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:36.189656 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a62b81-8e61-469c-ac6f-dc64b03a07c8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:36.283807 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1349/2655] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:36 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:36.289454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06d70c-6239-4fab-9b04-078e387771a4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:36.300702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06d70c-6239-4fab-9b04-078e387771a4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:36.306157 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a3ecc-e744-4c84-9be4-f77fa22f5571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:36.306811 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a3ecc-e744-4c84-9be4-f77fa22f5571 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:36.307100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a3ecc-e744-4c84-9be4-f77fa22f5571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:36.307377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a3ecc-e744-4c84-9be4-f77fa22f5571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:36.308084 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a3ecc-e744-4c84-9be4-f77fa22f5571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:36.309738 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a3ecc-e744-4c84-9be4-f77fa22f5571 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:36.320150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1307/2656] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:36.324315 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a62b81-8e61-469c-ac6f-dc64b03a07c8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:36.337445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a62b81-8e61-469c-ac6f-dc64b03a07c8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:36.340760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f27686-cb45-4c8c-ac39-c436d5ba8162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:36.341415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f27686-cb45-4c8c-ac39-c436d5ba8162 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:36.341707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f27686-cb45-4c8c-ac39-c436d5ba8162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:36.341992 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f27686-cb45-4c8c-ac39-c436d5ba8162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:36.342696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f27686-cb45-4c8c-ac39-c436d5ba8162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:36.344486 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f27686-cb45-4c8c-ac39-c436d5ba8162 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:36.357105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1350/2657] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:36.361868 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a3ecc-e744-4c84-9be4-f77fa22f5571 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:36.375853 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a3ecc-e744-4c84-9be4-f77fa22f5571 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:36.380807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b281b-e3b4-46b8-9e8b-72d0bf385d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:36.381530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b281b-e3b4-46b8-9e8b-72d0bf385d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:36.381982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b281b-e3b4-46b8-9e8b-72d0bf385d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:36.382483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b281b-e3b4-46b8-9e8b-72d0bf385d2c admin admin] PATH_INFO: `/v3/projects/2ef713b9f677486496652fe2aed14200/users/cb660a6fd8954c8e9358206fed90bf96/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:36.407906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b281b-e3b4-46b8-9e8b-72d0bf385d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef713b9f677486496652fe2aed14200, user_id=cb660a6fd8954c8e9358206fed90bf96, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:36.410641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b281b-e3b4-46b8-9e8b-72d0bf385d2c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:36.530697 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1308/2658] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:36 2022] PUT /identity/v3/projects/2ef713b9f677486496652fe2aed14200/users/cb660a6fd8954c8e9358206fed90bf96/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:36.541640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76134b5-f134-4b8c-8d55-f7298cd93a2a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:36.541956 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76134b5-f134-4b8c-8d55-f7298cd93a2a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:36.542242 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76134b5-f134-4b8c-8d55-f7298cd93a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:36.569670 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f76134b5-f134-4b8c-8d55-f7298cd93a2a None None] MFA Rules not processed for user `cb660a6fd8954c8e9358206fed90bf96`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:36.570876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f76134b5-f134-4b8c-8d55-f7298cd93a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:36.637353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1351/2659] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:36.641644 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b281b-e3b4-46b8-9e8b-72d0bf385d2c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:36.648193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:36.704220 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b281b-e3b4-46b8-9e8b-72d0bf385d2c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:36.709251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc307f4-b3ca-4f24-bf19-d9b1ec6fd034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:36.710473 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc307f4-b3ca-4f24-bf19-d9b1ec6fd034 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:36.710761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc307f4-b3ca-4f24-bf19-d9b1ec6fd034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:36.711036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc307f4-b3ca-4f24-bf19-d9b1ec6fd034 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:36.711739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc307f4-b3ca-4f24-bf19-d9b1ec6fd034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:36.713667 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc307f4-b3ca-4f24-bf19-d9b1ec6fd034 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:36.816948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1309/2660] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:36 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:36.821563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76134b5-f134-4b8c-8d55-f7298cd93a2a None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:36.832492 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76134b5-f134-4b8c-8d55-f7298cd93a2a None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:37.018399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8de88-cea9-4831-a627-5c05e3ddca36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:37.019033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8de88-cea9-4831-a627-5c05e3ddca36 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:37.019321 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8de88-cea9-4831-a627-5c05e3ddca36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:37.019640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8de88-cea9-4831-a627-5c05e3ddca36 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:37.020327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8de88-cea9-4831-a627-5c05e3ddca36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:37.022333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8de88-cea9-4831-a627-5c05e3ddca36 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:37.090893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1352/2661] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:36 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:37.095020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc307f4-b3ca-4f24-bf19-d9b1ec6fd034 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:37.106361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc307f4-b3ca-4f24-bf19-d9b1ec6fd034 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:37.113399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f068a-695b-4b3e-b5de-7109c173502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:37.113399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f068a-695b-4b3e-b5de-7109c173502a admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:37.113399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f068a-695b-4b3e-b5de-7109c173502a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:37.113399 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f068a-695b-4b3e-b5de-7109c173502a admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:37.113886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f068a-695b-4b3e-b5de-7109c173502a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:37.114548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f068a-695b-4b3e-b5de-7109c173502a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:37.126442 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1310/2662] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:37.130760 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8de88-cea9-4831-a627-5c05e3ddca36 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:37.140828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8de88-cea9-4831-a627-5c05e3ddca36 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:37.145602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56568e18-5385-410c-8344-e8055ecb71f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:37.146288 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56568e18-5385-410c-8344-e8055ecb71f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:37.146574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56568e18-5385-410c-8344-e8055ecb71f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:37.146852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56568e18-5385-410c-8344-e8055ecb71f6 admin admin] PATH_INFO: `/v3/projects/3df753c5cf5543c099bc727131e0d18f/users/0beda7bc43be407696c720ba86a92679/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:37.162200 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56568e18-5385-410c-8344-e8055ecb71f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df753c5cf5543c099bc727131e0d18f, user_id=0beda7bc43be407696c720ba86a92679, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:37.165540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56568e18-5385-410c-8344-e8055ecb71f6 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:37.226493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1353/2663] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:37 2022] PUT /identity/v3/projects/3df753c5cf5543c099bc727131e0d18f/users/0beda7bc43be407696c720ba86a92679/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:37.230630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f068a-695b-4b3e-b5de-7109c173502a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:37.242365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f068a-695b-4b3e-b5de-7109c173502a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:37.299878 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b704ae-9754-4246-9803-20c6eaf6f8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:37.300503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b704ae-9754-4246-9803-20c6eaf6f8fc admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:37.300787 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b704ae-9754-4246-9803-20c6eaf6f8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:37.301065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b704ae-9754-4246-9803-20c6eaf6f8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:37.301800 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b704ae-9754-4246-9803-20c6eaf6f8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:37.303427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b704ae-9754-4246-9803-20c6eaf6f8fc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:37.313417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1311/2664] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:37.317818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56568e18-5385-410c-8344-e8055ecb71f6 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:37.328224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56568e18-5385-410c-8344-e8055ecb71f6 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:37.333350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6b300-2d72-4007-92d9-d96907311909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:37.334003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6b300-2d72-4007-92d9-d96907311909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:37.334289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6b300-2d72-4007-92d9-d96907311909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:37.335094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6b300-2d72-4007-92d9-d96907311909 admin admin] PATH_INFO: `/v3/projects/3df753c5cf5543c099bc727131e0d18f/users/0beda7bc43be407696c720ba86a92679/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:37.337330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6b300-2d72-4007-92d9-d96907311909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df753c5cf5543c099bc727131e0d18f, user_id=0beda7bc43be407696c720ba86a92679, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:37.340030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6b300-2d72-4007-92d9-d96907311909 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:37.345856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:37.352474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1354/2665] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:37 2022] PUT /identity/v3/projects/3df753c5cf5543c099bc727131e0d18f/users/0beda7bc43be407696c720ba86a92679/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:37.363582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f808f-28fe-4da4-a67c-c327838817ff None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:37.363873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f808f-28fe-4da4-a67c-c327838817ff None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:37.364156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f808f-28fe-4da4-a67c-c327838817ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:37.402782 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-991f808f-28fe-4da4-a67c-c327838817ff None None] MFA Rules not processed for user `0beda7bc43be407696c720ba86a92679`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:37.404114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-991f808f-28fe-4da4-a67c-c327838817ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:37.473210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:37.477641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1312/2666] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:37.489027 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6b300-2d72-4007-92d9-d96907311909 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:37.490730 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-69f6b300-2d72-4007-92d9-d96907311909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:37.552487 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6b300-2d72-4007-92d9-d96907311909 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:37.556806 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e80d-b38f-43c9-a22d-d3088095ec63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:37.557532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e80d-b38f-43c9-a22d-d3088095ec63 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:37.557908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e80d-b38f-43c9-a22d-d3088095ec63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:37.558322 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e80d-b38f-43c9-a22d-d3088095ec63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:37.568012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e80d-b38f-43c9-a22d-d3088095ec63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:37.570440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e80d-b38f-43c9-a22d-d3088095ec63 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:37.580408 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1355/2667] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:37.806576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f808f-28fe-4da4-a67c-c327838817ff None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:37.816650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f808f-28fe-4da4-a67c-c327838817ff None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:37.821826 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb70bb-b8bb-4989-9820-6d6d897c0c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:37.822458 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb70bb-b8bb-4989-9820-6d6d897c0c5e service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:37.822744 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb70bb-b8bb-4989-9820-6d6d897c0c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:37.823024 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb70bb-b8bb-4989-9820-6d6d897c0c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:37.887735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb70bb-b8bb-4989-9820-6d6d897c0c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:37.889193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb70bb-b8bb-4989-9820-6d6d897c0c5e service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:37.898818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1313/2668] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:45.947811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e80d-b38f-43c9-a22d-d3088095ec63 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:45.949233 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b9a0e80d-b38f-43c9-a22d-d3088095ec63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:46.016409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e80d-b38f-43c9-a22d-d3088095ec63 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:46.021343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:46.021991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:46.022279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:46.022560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] PATH_INFO: `/v3/users/0c7d93ae05024369ab4c30270d9f0ac4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:46.023902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c7d93ae05024369ab4c30270d9f0ac4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:46.025861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:46.245504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c7d93ae05024369ab4c30270d9f0ac4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:46.257430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c7d93ae05024369ab4c30270d9f0ac4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:46.258707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c7d93ae05024369ab4c30270d9f0ac4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:46.265586 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c7d93ae05024369ab4c30270d9f0ac4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:46.364470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c7d93ae05024369ab4c30270d9f0ac4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:46.375311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1356/2669] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:45 2022] DELETE /identity/v3/users/0c7d93ae05024369ab4c30270d9f0ac4 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:46.377258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875328fc-f548-45ca-a18f-614eae477cde None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:46.377733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875328fc-f548-45ca-a18f-614eae477cde None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:46.377915 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875328fc-f548-45ca-a18f-614eae477cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:46.399610 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-875328fc-f548-45ca-a18f-614eae477cde None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:46.401509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-875328fc-f548-45ca-a18f-614eae477cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:46.465325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1314/2670] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:46.475279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:46.477429 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:46.500359 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:46.539181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0f7b1-59e8-47f7-a9d7-2bdaebf932d3 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:46.543301 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f5662-760c-4963-ba16-a1d1a2c7b913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:46.551786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f5662-760c-4963-ba16-a1d1a2c7b913 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:46.551786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f5662-760c-4963-ba16-a1d1a2c7b913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:46.551786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f5662-760c-4963-ba16-a1d1a2c7b913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:46.554884 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f5662-760c-4963-ba16-a1d1a2c7b913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:46.556351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f5662-760c-4963-ba16-a1d1a2c7b913 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:46.566562 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1357/2671] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:46.992971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875328fc-f548-45ca-a18f-614eae477cde None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:46.994552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-875328fc-f548-45ca-a18f-614eae477cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:47.006565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875328fc-f548-45ca-a18f-614eae477cde None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:47.011467 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:47.012146 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:47.012426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:47.012747 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] PATH_INFO: `/v3/projects/8ac6674505224cb58374b5976f14167b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:47.014054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac6674505224cb58374b5976f14167b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:47.015982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:47.102620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac6674505224cb58374b5976f14167b', 'request_id': 'req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:47.108367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac6674505224cb58374b5976f14167b', 'request_id': 'req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:47.114641 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac6674505224cb58374b5976f14167b', 'request_id': 'req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:47.179019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] The token cache is being invalidate because project 8ac6674505224cb58374b5976f14167b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:47.179302 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:47.180032 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:47.181591 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1315/2672] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:46 2022] DELETE /identity/v3/projects/8ac6674505224cb58374b5976f14167b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:47.185711 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f5662-760c-4963-ba16-a1d1a2c7b913 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:47.195400 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f5662-760c-4963-ba16-a1d1a2c7b913 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:47.252284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:47.252902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:47.253186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:47.253494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] PATH_INFO: `/v3/users/9694f4f0bd6848c2906acb1f0009e221` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:47.254665 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9694f4f0bd6848c2906acb1f0009e221)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:47.256564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:47.499778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9694f4f0bd6848c2906acb1f0009e221'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:47.512950 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9694f4f0bd6848c2906acb1f0009e221'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:47.513820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9694f4f0bd6848c2906acb1f0009e221'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:47.521862 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9694f4f0bd6848c2906acb1f0009e221'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:47.591526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-51fcca4c-cf9d-42cc-9eee-a780f6e3f854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9694f4f0bd6848c2906acb1f0009e221'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:47.597909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1358/2673] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:47 2022] DELETE /identity/v3/users/9694f4f0bd6848c2906acb1f0009e221 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:48.036123 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:48.037677 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:48.050643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83018ffc-c9b7-484b-9f11-ebf27e72fc0a admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:48.069151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:48.109050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:48.109721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:48.110003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:48.110284 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] PATH_INFO: `/v3/projects/d68baa42375b44038328b82c8243c069` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:48.111460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d68baa42375b44038328b82c8243c069)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:48.113486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:48.239087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd68baa42375b44038328b82c8243c069', 'request_id': 'req-972b2e2b-c559-4c88-b59c-731bbdf996cf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:48.244923 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd68baa42375b44038328b82c8243c069', 'request_id': 'req-972b2e2b-c559-4c88-b59c-731bbdf996cf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:48.251607 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd68baa42375b44038328b82c8243c069', 'request_id': 'req-972b2e2b-c559-4c88-b59c-731bbdf996cf'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:48.353689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] The token cache is being invalidate because project d68baa42375b44038328b82c8243c069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:48.353689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:48.353689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-972b2e2b-c559-4c88-b59c-731bbdf996cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:48.355814 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1316/2674] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:48 2022] DELETE /identity/v3/projects/d68baa42375b44038328b82c8243c069 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:48.359418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd0a2-ca17-409b-834b-fbc376ba4e47 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:48.359921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd0a2-ca17-409b-834b-fbc376ba4e47 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:48.360008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd0a2-ca17-409b-834b-fbc376ba4e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:48.383263 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-8d0dd0a2-ca17-409b-834b-fbc376ba4e47 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:48.384589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8d0dd0a2-ca17-409b-834b-fbc376ba4e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:48.425833 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:48.447660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1359/2675] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:48.455746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e053564-726c-4371-a108-a3d08dbaebd1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:48.456051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e053564-726c-4371-a108-a3d08dbaebd1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:48.456329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e053564-726c-4371-a108-a3d08dbaebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:48.476507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6e053564-726c-4371-a108-a3d08dbaebd1 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:48.477925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6e053564-726c-4371-a108-a3d08dbaebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:48.485909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1317/2676] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:48.490695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dd0a2-ca17-409b-834b-fbc376ba4e47 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:48.501822 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dd0a2-ca17-409b-834b-fbc376ba4e47 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:48.506633 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fa1b7-4d7f-44c9-990f-58de3b4e41b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:48.507255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fa1b7-4d7f-44c9-990f-58de3b4e41b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:48.508210 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fa1b7-4d7f-44c9-990f-58de3b4e41b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:48.508547 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fa1b7-4d7f-44c9-990f-58de3b4e41b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:48.509279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fa1b7-4d7f-44c9-990f-58de3b4e41b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:48.509620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fa1b7-4d7f-44c9-990f-58de3b4e41b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:48.511234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fa1b7-4d7f-44c9-990f-58de3b4e41b5 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:48.525186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1360/2677] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:36:48.534288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e053564-726c-4371-a108-a3d08dbaebd1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:48.543942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e053564-726c-4371-a108-a3d08dbaebd1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:48.549083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01273f6-327d-48e1-be87-b23e434dd450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:48.549745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01273f6-327d-48e1-be87-b23e434dd450 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:48.550054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01273f6-327d-48e1-be87-b23e434dd450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:48.550333 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01273f6-327d-48e1-be87-b23e434dd450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:48.551022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01273f6-327d-48e1-be87-b23e434dd450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:48.552994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01273f6-327d-48e1-be87-b23e434dd450 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:48.692324 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1318/2678] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:48 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:48.693413 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fa1b7-4d7f-44c9-990f-58de3b4e41b5 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:48.704255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fa1b7-4d7f-44c9-990f-58de3b4e41b5 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:48.771295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cfca3-369e-4b0d-85e7-d07f1ed8fc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:48.771929 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cfca3-369e-4b0d-85e7-d07f1ed8fc73 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:48.772219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cfca3-369e-4b0d-85e7-d07f1ed8fc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:48.772498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cfca3-369e-4b0d-85e7-d07f1ed8fc73 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:48.773849 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713cfca3-369e-4b0d-85e7-d07f1ed8fc73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:48.775866 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713cfca3-369e-4b0d-85e7-d07f1ed8fc73 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:48.912755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1361/2679] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:48 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:48.916831 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01273f6-327d-48e1-be87-b23e434dd450 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:48.929636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01273f6-327d-48e1-be87-b23e434dd450 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:48.934484 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147594c3-a047-4f43-9ff8-96953e21bf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:48.935242 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147594c3-a047-4f43-9ff8-96953e21bf43 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:48.935535 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147594c3-a047-4f43-9ff8-96953e21bf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:48.935813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147594c3-a047-4f43-9ff8-96953e21bf43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:48.936517 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147594c3-a047-4f43-9ff8-96953e21bf43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:48.938170 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147594c3-a047-4f43-9ff8-96953e21bf43 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:48.949410 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1319/2680] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:48.958646 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cfca3-369e-4b0d-85e7-d07f1ed8fc73 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:48.970719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cfca3-369e-4b0d-85e7-d07f1ed8fc73 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:48.975925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b29ed8-9c23-495b-843c-bd102f076306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:48.976783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29ed8-9c23-495b-843c-bd102f076306 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:48.977133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29ed8-9c23-495b-843c-bd102f076306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:48.977467 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29ed8-9c23-495b-843c-bd102f076306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:48.978130 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b29ed8-9c23-495b-843c-bd102f076306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:48.979735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b29ed8-9c23-495b-843c-bd102f076306 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:48.990675 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1362/2681] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:48.995327 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147594c3-a047-4f43-9ff8-96953e21bf43 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:49.005321 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147594c3-a047-4f43-9ff8-96953e21bf43 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:49.010539 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdf77b-b5ca-40c8-86b5-d4a054822c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:49.011131 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdf77b-b5ca-40c8-86b5-d4a054822c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:49.011420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdf77b-b5ca-40c8-86b5-d4a054822c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:49.011696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdf77b-b5ca-40c8-86b5-d4a054822c85 admin admin] PATH_INFO: `/v3/projects/d83296b876c7490789ecfed13f41d643/users/3a4681d4199743cd972a00520f3f70e3/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:49.028104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fdf77b-b5ca-40c8-86b5-d4a054822c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83296b876c7490789ecfed13f41d643, user_id=3a4681d4199743cd972a00520f3f70e3, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:49.030808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fdf77b-b5ca-40c8-86b5-d4a054822c85 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:49.112213 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1320/2682] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:48 2022] PUT /identity/v3/projects/d83296b876c7490789ecfed13f41d643/users/3a4681d4199743cd972a00520f3f70e3/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:49.124284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d1c17-937d-42af-ba38-9ddadf30b7aa None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:49.124522 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d1c17-937d-42af-ba38-9ddadf30b7aa None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:49.124790 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d1c17-937d-42af-ba38-9ddadf30b7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:49.154870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-439d1c17-937d-42af-ba38-9ddadf30b7aa None None] MFA Rules not processed for user `3a4681d4199743cd972a00520f3f70e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:49.156096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-439d1c17-937d-42af-ba38-9ddadf30b7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:49.227652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1363/2683] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:49.232363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdf77b-b5ca-40c8-86b5-d4a054822c85 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:49.299506 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:49.314005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdf77b-b5ca-40c8-86b5-d4a054822c85 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:49.319164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43441113-11b1-4f4b-89c8-5cca7ca70e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:49.319838 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43441113-11b1-4f4b-89c8-5cca7ca70e71 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:49.320803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43441113-11b1-4f4b-89c8-5cca7ca70e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:49.321332 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43441113-11b1-4f4b-89c8-5cca7ca70e71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:49.322072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43441113-11b1-4f4b-89c8-5cca7ca70e71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:49.324035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43441113-11b1-4f4b-89c8-5cca7ca70e71 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:49.436379 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1321/2684] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:49 2022] POST /identity/v3/projects => generated 379 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:49.440555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d1c17-937d-42af-ba38-9ddadf30b7aa None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:49.453099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d1c17-937d-42af-ba38-9ddadf30b7aa None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:49.487270 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:49.511825 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018d984-8f5e-4876-bf8d-d5659d9aa791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:49.512447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018d984-8f5e-4876-bf8d-d5659d9aa791 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:49.512731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018d984-8f5e-4876-bf8d-d5659d9aa791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:49.513010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018d984-8f5e-4876-bf8d-d5659d9aa791 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:49.513717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018d984-8f5e-4876-bf8d-d5659d9aa791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:49.515847 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018d984-8f5e-4876-bf8d-d5659d9aa791 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:49.610968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1364/2685] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:49 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:49.615261 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43441113-11b1-4f4b-89c8-5cca7ca70e71 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:49.626345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43441113-11b1-4f4b-89c8-5cca7ca70e71 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:49.631163 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79460e-f43b-42a8-b712-9d6da8fb0fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:49.631773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79460e-f43b-42a8-b712-9d6da8fb0fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:49.632060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79460e-f43b-42a8-b712-9d6da8fb0fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:49.632340 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79460e-f43b-42a8-b712-9d6da8fb0fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:49.633060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79460e-f43b-42a8-b712-9d6da8fb0fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:49.634757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79460e-f43b-42a8-b712-9d6da8fb0fb8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:49.645926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1322/2686] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:49.650196 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018d984-8f5e-4876-bf8d-d5659d9aa791 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:49.660550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018d984-8f5e-4876-bf8d-d5659d9aa791 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:49.666137 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a29a7-0179-493c-b61b-0d6c4607ad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:49.666808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a29a7-0179-493c-b61b-0d6c4607ad8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:49.667097 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a29a7-0179-493c-b61b-0d6c4607ad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:49.667383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a29a7-0179-493c-b61b-0d6c4607ad8b admin admin] PATH_INFO: `/v3/projects/6c7deed131bd45f49b0ba187f9f45dc6/users/b628d201e77441b3908a008362c84fc4/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:49.682907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a29a7-0179-493c-b61b-0d6c4607ad8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c7deed131bd45f49b0ba187f9f45dc6, user_id=b628d201e77441b3908a008362c84fc4, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:49.685623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a29a7-0179-493c-b61b-0d6c4607ad8b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:49.737171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1365/2687] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:49 2022] PUT /identity/v3/projects/6c7deed131bd45f49b0ba187f9f45dc6/users/b628d201e77441b3908a008362c84fc4/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:49.741318 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79460e-f43b-42a8-b712-9d6da8fb0fb8 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:49.752014 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79460e-f43b-42a8-b712-9d6da8fb0fb8 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:49.810227 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04430cf-6b32-45f8-92e5-77c69df6ed46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:49.811193 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04430cf-6b32-45f8-92e5-77c69df6ed46 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:49.811530 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04430cf-6b32-45f8-92e5-77c69df6ed46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:49.811933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04430cf-6b32-45f8-92e5-77c69df6ed46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:49.813287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04430cf-6b32-45f8-92e5-77c69df6ed46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:49.814996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04430cf-6b32-45f8-92e5-77c69df6ed46 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:49.826280 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1323/2688] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:49.830484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a29a7-0179-493c-b61b-0d6c4607ad8b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:49.842474 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a29a7-0179-493c-b61b-0d6c4607ad8b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:49.846996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0abf1b-f565-40de-a3fd-4a719b62e821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:49.847655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0abf1b-f565-40de-a3fd-4a719b62e821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:49.847937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0abf1b-f565-40de-a3fd-4a719b62e821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:49.848211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0abf1b-f565-40de-a3fd-4a719b62e821 admin admin] PATH_INFO: `/v3/projects/6c7deed131bd45f49b0ba187f9f45dc6/users/b628d201e77441b3908a008362c84fc4/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:49.850198 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0abf1b-f565-40de-a3fd-4a719b62e821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c7deed131bd45f49b0ba187f9f45dc6, user_id=b628d201e77441b3908a008362c84fc4, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:49.853611 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0abf1b-f565-40de-a3fd-4a719b62e821 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:49.865789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1366/2689] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:49 2022] PUT /identity/v3/projects/6c7deed131bd45f49b0ba187f9f45dc6/users/b628d201e77441b3908a008362c84fc4/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:49.876439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51061310-563f-484a-90cb-e638bce88fd5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:49.876735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51061310-563f-484a-90cb-e638bce88fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:49.877071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51061310-563f-484a-90cb-e638bce88fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:49.905654 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-51061310-563f-484a-90cb-e638bce88fd5 None None] MFA Rules not processed for user `b628d201e77441b3908a008362c84fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:49.906873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-51061310-563f-484a-90cb-e638bce88fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:49.975451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1324/2690] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:49.986865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0abf1b-f565-40de-a3fd-4a719b62e821 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:49.988620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ff0abf1b-f565-40de-a3fd-4a719b62e821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:50.049682 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0abf1b-f565-40de-a3fd-4a719b62e821 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:50.053405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddce159-36d8-4a05-9273-d004518095c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:50.058928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddce159-36d8-4a05-9273-d004518095c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:50.058928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddce159-36d8-4a05-9273-d004518095c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:50.058928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddce159-36d8-4a05-9273-d004518095c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:50.064084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddce159-36d8-4a05-9273-d004518095c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:50.065540 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddce159-36d8-4a05-9273-d004518095c7 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:50.075211 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1367/2691] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:36:50.912646 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51061310-563f-484a-90cb-e638bce88fd5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:50.968429 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:50.990167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51061310-563f-484a-90cb-e638bce88fd5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:50.995023 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:50.995672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:50.995953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:50.996228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] PATH_INFO: `/v3/users/3a4681d4199743cd972a00520f3f70e3` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:50.997453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a4681d4199743cd972a00520f3f70e3)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:50.999356 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:51.217716 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a4681d4199743cd972a00520f3f70e3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:51.229906 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a4681d4199743cd972a00520f3f70e3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:51.236909 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a4681d4199743cd972a00520f3f70e3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:51.322943 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a4681d4199743cd972a00520f3f70e3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:51.323587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a4681d4199743cd972a00520f3f70e3'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:51.330315 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1325/2692] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:50 2022] DELETE /identity/v3/users/3a4681d4199743cd972a00520f3f70e3 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:51.335383 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e9902-1d0a-4253-a477-ffe1a6a4a257 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:51.336508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e9902-1d0a-4253-a477-ffe1a6a4a257 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:51.336711 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e9902-1d0a-4253-a477-ffe1a6a4a257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:51.356965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-921e9902-1d0a-4253-a477-ffe1a6a4a257 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:51.358286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-921e9902-1d0a-4253-a477-ffe1a6a4a257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:51.418888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1368/2693] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:51.428501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:51.430135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:51.490862 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476c52e-ccf1-493b-bbe2-d7618e47d2a4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:51.494623 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5b96b-0a9c-42f9-a931-f33712c24cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:51.495248 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5b96b-0a9c-42f9-a931-f33712c24cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:51.495536 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5b96b-0a9c-42f9-a931-f33712c24cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:51.495837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5b96b-0a9c-42f9-a931-f33712c24cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:51.505140 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5b96b-0a9c-42f9-a931-f33712c24cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:51.506614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5b96b-0a9c-42f9-a931-f33712c24cd1 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:51.516204 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1326/2694] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:51.980901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e9902-1d0a-4253-a477-ffe1a6a4a257 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:51.980901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-921e9902-1d0a-4253-a477-ffe1a6a4a257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:51.992969 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e9902-1d0a-4253-a477-ffe1a6a4a257 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:51.998031 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:51.998548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:51.998841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:51.999131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] PATH_INFO: `/v3/projects/d83296b876c7490789ecfed13f41d643` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:52.000311 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d83296b876c7490789ecfed13f41d643)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:52.002351 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:52.105008 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:52.121318 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd83296b876c7490789ecfed13f41d643', 'request_id': 'req-da499588-8128-4fa1-b043-3b01d4b40324'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:52.126591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd83296b876c7490789ecfed13f41d643', 'request_id': 'req-da499588-8128-4fa1-b043-3b01d4b40324'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:52.187472 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd83296b876c7490789ecfed13f41d643', 'request_id': 'req-da499588-8128-4fa1-b043-3b01d4b40324'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:52.200565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] The token cache is being invalidate because project d83296b876c7490789ecfed13f41d643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:52.200854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:52.201519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:52.202996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1369/2695] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:51 2022] DELETE /identity/v3/projects/d83296b876c7490789ecfed13f41d643 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:52.207029 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5b96b-0a9c-42f9-a931-f33712c24cd1 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:52.218091 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5b96b-0a9c-42f9-a931-f33712c24cd1 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:52.276440 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:52.277104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:52.277425 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:52.283438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] PATH_INFO: `/v3/users/b628d201e77441b3908a008362c84fc4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:52.283438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b628d201e77441b3908a008362c84fc4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:52.283438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:52.286656 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:52.470111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b628d201e77441b3908a008362c84fc4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:52.482562 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b628d201e77441b3908a008362c84fc4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:52.489294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b628d201e77441b3908a008362c84fc4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:52.542102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b628d201e77441b3908a008362c84fc4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:52.542688 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b628d201e77441b3908a008362c84fc4'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:52.549783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1327/2696] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:52 2022] DELETE /identity/v3/users/b628d201e77441b3908a008362c84fc4 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:52.944746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:52.947451 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:52.963405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da499588-8128-4fa1-b043-3b01d4b40324 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:53.028131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:53.028791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.029084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.029454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] PATH_INFO: `/v3/projects/6c7deed131bd45f49b0ba187f9f45dc6` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.035505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c7deed131bd45f49b0ba187f9f45dc6)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:53.035505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:53.151810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:53.153219 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:53.166846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64ff43-3488-48a5-a25e-39a61ddf98dd admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:53.171595 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:53.178090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.178090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.178090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] PATH_INFO: `/v3/users/cb660a6fd8954c8e9358206fed90bf96` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.178090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb660a6fd8954c8e9358206fed90bf96)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:53.178090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:53.194280 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c7deed131bd45f49b0ba187f9f45dc6', 'request_id': 'req-2bea9500-4527-44ad-b907-277a06e6b8ee'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:53.199900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c7deed131bd45f49b0ba187f9f45dc6', 'request_id': 'req-2bea9500-4527-44ad-b907-277a06e6b8ee'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:53.290991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c7deed131bd45f49b0ba187f9f45dc6', 'request_id': 'req-2bea9500-4527-44ad-b907-277a06e6b8ee'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:53.303006 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] The token cache is being invalidate because project 6c7deed131bd45f49b0ba187f9f45dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:53.304076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:53.304076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-2bea9500-4527-44ad-b907-277a06e6b8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:53.305615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1370/2697] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:52 2022] DELETE /identity/v3/projects/6c7deed131bd45f49b0ba187f9f45dc6 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:53.310265 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539383cc-f5ee-4b59-974d-c425b30e1488 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.310561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539383cc-f5ee-4b59-974d-c425b30e1488 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.310843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539383cc-f5ee-4b59-974d-c425b30e1488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.330672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-539383cc-f5ee-4b59-974d-c425b30e1488 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:53.331882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-539383cc-f5ee-4b59-974d-c425b30e1488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:53.374525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:53.393374 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1371/2698] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:53.403087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf25c4c-31ee-46e7-8134-caee16b2c328 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.410235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf25c4c-31ee-46e7-8134-caee16b2c328 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.410235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf25c4c-31ee-46e7-8134-caee16b2c328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.414913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb660a6fd8954c8e9358206fed90bf96'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:53.425022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-6bf25c4c-31ee-46e7-8134-caee16b2c328 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:53.426262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6bf25c4c-31ee-46e7-8134-caee16b2c328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:53.427752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb660a6fd8954c8e9358206fed90bf96'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:53.435076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb660a6fd8954c8e9358206fed90bf96'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:53.435660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1372/2699] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:53.439612 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf25c4c-31ee-46e7-8134-caee16b2c328 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:53.449245 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf25c4c-31ee-46e7-8134-caee16b2c328 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:53.454306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532443d-a9d5-495a-9ca5-10ab3ab5d1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:53.454920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532443d-a9d5-495a-9ca5-10ab3ab5d1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.457534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532443d-a9d5-495a-9ca5-10ab3ab5d1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.457534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532443d-a9d5-495a-9ca5-10ab3ab5d1d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.457534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532443d-a9d5-495a-9ca5-10ab3ab5d1d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:53.457534 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532443d-a9d5-495a-9ca5-10ab3ab5d1d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:53.458440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532443d-a9d5-495a-9ca5-10ab3ab5d1d7 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:53.470585 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1373/2700] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:36:53.474906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532443d-a9d5-495a-9ca5-10ab3ab5d1d7 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:53.484815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532443d-a9d5-495a-9ca5-10ab3ab5d1d7 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:53.489471 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d432fd4-515a-4d71-8a54-b31557a1c1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:53.490063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d432fd4-515a-4d71-8a54-b31557a1c1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.490342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d432fd4-515a-4d71-8a54-b31557a1c1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.490615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d432fd4-515a-4d71-8a54-b31557a1c1e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.491267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d432fd4-515a-4d71-8a54-b31557a1c1e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:53.493246 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d432fd4-515a-4d71-8a54-b31557a1c1e1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:53.500070 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb660a6fd8954c8e9358206fed90bf96'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:53.500780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-285b4157-47ff-4ba2-8923-be96ff3d8ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb660a6fd8954c8e9358206fed90bf96'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:53.508165 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1328/2701] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:53 2022] DELETE /identity/v3/users/cb660a6fd8954c8e9358206fed90bf96 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:53.512580 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897c279-da98-430a-8cb6-03ec979ed776 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.512930 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897c279-da98-430a-8cb6-03ec979ed776 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.513218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897c279-da98-430a-8cb6-03ec979ed776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.532894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8897c279-da98-430a-8cb6-03ec979ed776 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:53.534228 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8897c279-da98-430a-8cb6-03ec979ed776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:53.586109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:53.591647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1374/2702] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:53 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:53.592637 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d432fd4-515a-4d71-8a54-b31557a1c1e1 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:53.594251 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-1d432fd4-515a-4d71-8a54-b31557a1c1e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:53.595746 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1329/2703] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:53.607162 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897c279-da98-430a-8cb6-03ec979ed776 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:53.608588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8897c279-da98-430a-8cb6-03ec979ed776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:53.666353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d432fd4-515a-4d71-8a54-b31557a1c1e1 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:53.668139 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897c279-da98-430a-8cb6-03ec979ed776 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:53.670961 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c0c35-ae9b-4b56-8f94-1f7625893f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:53.671582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c0c35-ae9b-4b56-8f94-1f7625893f64 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.671903 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c0c35-ae9b-4b56-8f94-1f7625893f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.672192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c0c35-ae9b-4b56-8f94-1f7625893f64 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.672843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c0c35-ae9b-4b56-8f94-1f7625893f64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:53.674805 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c0c35-ae9b-4b56-8f94-1f7625893f64 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:53.720089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995e454-0f01-4fc4-bf46-339ab792ed8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:53.720718 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995e454-0f01-4fc4-bf46-339ab792ed8e service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.721006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995e454-0f01-4fc4-bf46-339ab792ed8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.721287 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995e454-0f01-4fc4-bf46-339ab792ed8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.730991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995e454-0f01-4fc4-bf46-339ab792ed8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:53.732448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995e454-0f01-4fc4-bf46-339ab792ed8e service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:53.742624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1330/2704] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:53.768075 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1375/2705] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:53 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:53.772606 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995e454-0f01-4fc4-bf46-339ab792ed8e service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:53.784151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995e454-0f01-4fc4-bf46-339ab792ed8e service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:53.790094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec4397-97e6-4e7e-b4eb-ee90c4f9d24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:53.790774 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec4397-97e6-4e7e-b4eb-ee90c4f9d24d admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.791067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec4397-97e6-4e7e-b4eb-ee90c4f9d24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.791352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec4397-97e6-4e7e-b4eb-ee90c4f9d24d admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.792091 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec4397-97e6-4e7e-b4eb-ee90c4f9d24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:53.793770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec4397-97e6-4e7e-b4eb-ee90c4f9d24d admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:53.805068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1331/2706] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:53.809428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c0c35-ae9b-4b56-8f94-1f7625893f64 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:53.820093 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c0c35-ae9b-4b56-8f94-1f7625893f64 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:53.825570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c97d9-51d0-4a4c-b841-f6f76454ac72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:53.826295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c97d9-51d0-4a4c-b841-f6f76454ac72 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.831927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c97d9-51d0-4a4c-b841-f6f76454ac72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.831927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c97d9-51d0-4a4c-b841-f6f76454ac72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.831927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c97d9-51d0-4a4c-b841-f6f76454ac72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:53.831927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c97d9-51d0-4a4c-b841-f6f76454ac72 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:53.841370 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1376/2707] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:53.845559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec4397-97e6-4e7e-b4eb-ee90c4f9d24d admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:53.860937 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec4397-97e6-4e7e-b4eb-ee90c4f9d24d admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:53.865842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5792a39-0b36-4fd2-8bb8-a54972912cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:53.866989 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5792a39-0b36-4fd2-8bb8-a54972912cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.867508 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5792a39-0b36-4fd2-8bb8-a54972912cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.867969 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5792a39-0b36-4fd2-8bb8-a54972912cbc admin admin] PATH_INFO: `/v3/projects/fb96b2d8474943238c0939d42e492d14/users/ec5b19159e8047fcb646b49416f926fd/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:53.883387 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5792a39-0b36-4fd2-8bb8-a54972912cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb96b2d8474943238c0939d42e492d14, user_id=ec5b19159e8047fcb646b49416f926fd, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:53.886077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5792a39-0b36-4fd2-8bb8-a54972912cbc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:53.983252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1332/2708] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:53 2022] PUT /identity/v3/projects/fb96b2d8474943238c0939d42e492d14/users/ec5b19159e8047fcb646b49416f926fd/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:53.997477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc28905-026b-4298-b4d5-872129f95971 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:53.998076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc28905-026b-4298-b4d5-872129f95971 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:53.998076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc28905-026b-4298-b4d5-872129f95971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:54.029581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-ddc28905-026b-4298-b4d5-872129f95971 None None] MFA Rules not processed for user `ec5b19159e8047fcb646b49416f926fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:54.031262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ddc28905-026b-4298-b4d5-872129f95971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:54.099809 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1377/2709] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:54.104658 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5792a39-0b36-4fd2-8bb8-a54972912cbc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:54.166701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5792a39-0b36-4fd2-8bb8-a54972912cbc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:54.171682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e795de-7d0a-425d-bda7-f5aa5d94088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:54.172237 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e795de-7d0a-425d-bda7-f5aa5d94088e admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:54.172520 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e795de-7d0a-425d-bda7-f5aa5d94088e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:54.172800 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e795de-7d0a-425d-bda7-f5aa5d94088e admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:54.173648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e795de-7d0a-425d-bda7-f5aa5d94088e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:54.175639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e795de-7d0a-425d-bda7-f5aa5d94088e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:54.190267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc28905-026b-4298-b4d5-872129f95971 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:54.194549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-ddc28905-026b-4298-b4d5-872129f95971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:54.207479 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc28905-026b-4298-b4d5-872129f95971 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:54.213579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:54.214236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:54.214521 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:54.214860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] PATH_INFO: `/v3/projects/2ef713b9f677486496652fe2aed14200` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:54.216065 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef713b9f677486496652fe2aed14200)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:54.218078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:54.227845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:54.275984 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1333/2710] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:54 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:54.280000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e795de-7d0a-425d-bda7-f5aa5d94088e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:54.289427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:54.342777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e795de-7d0a-425d-bda7-f5aa5d94088e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:54.347927 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80aee3-8fe9-418c-af2d-5eb984d0f037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:54.348649 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80aee3-8fe9-418c-af2d-5eb984d0f037 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:54.348898 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80aee3-8fe9-418c-af2d-5eb984d0f037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:54.349364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80aee3-8fe9-418c-af2d-5eb984d0f037 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:54.350057 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80aee3-8fe9-418c-af2d-5eb984d0f037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:54.352728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80aee3-8fe9-418c-af2d-5eb984d0f037 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:54.379951 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef713b9f677486496652fe2aed14200', 'request_id': 'req-46ccc613-3ddf-4394-8761-f1199925525e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:54.385953 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef713b9f677486496652fe2aed14200', 'request_id': 'req-46ccc613-3ddf-4394-8761-f1199925525e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:54.481288 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1334/2711] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:54 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:54.485788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80aee3-8fe9-418c-af2d-5eb984d0f037 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:54.495737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80aee3-8fe9-418c-af2d-5eb984d0f037 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:54.500079 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef713b9f677486496652fe2aed14200', 'request_id': 'req-46ccc613-3ddf-4394-8761-f1199925525e'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:54.509905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] The token cache is being invalidate because project 2ef713b9f677486496652fe2aed14200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:54.510207 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:54.510874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:54.512396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1378/2712] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:54 2022] DELETE /identity/v3/projects/2ef713b9f677486496652fe2aed14200 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:54.516254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:54.518671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:54.554575 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6df6c-12ff-4d52-a525-ca8d2df18940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:54.555212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6df6c-12ff-4d52-a525-ca8d2df18940 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:54.555503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6df6c-12ff-4d52-a525-ca8d2df18940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:54.555788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6df6c-12ff-4d52-a525-ca8d2df18940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:54.556496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6df6c-12ff-4d52-a525-ca8d2df18940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:54.558362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6df6c-12ff-4d52-a525-ca8d2df18940 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:54.568492 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1335/2713] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:54.572689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6df6c-12ff-4d52-a525-ca8d2df18940 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:54.574466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-11c6df6c-12ff-4d52-a525-ca8d2df18940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:54.584274 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccc613-3ddf-4394-8761-f1199925525e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:54.589748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:54.589979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6df6c-12ff-4d52-a525-ca8d2df18940 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:54.590418 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:54.590701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:54.590978 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] PATH_INFO: `/v3/users/0beda7bc43be407696c720ba86a92679` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:54.592267 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0beda7bc43be407696c720ba86a92679)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:54.595060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:54.600464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4df536-6750-4a09-adbd-f26ca44edc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:54.601025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4df536-6750-4a09-adbd-f26ca44edc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:54.602003 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4df536-6750-4a09-adbd-f26ca44edc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:54.602212 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4df536-6750-4a09-adbd-f26ca44edc9b admin admin] PATH_INFO: `/v3/projects/83bd52dbd811457c80f9218ab51a2255/users/341db551fa3e4c2ea98b16ea3ad991d5/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:54.617347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4df536-6750-4a09-adbd-f26ca44edc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=83bd52dbd811457c80f9218ab51a2255, user_id=341db551fa3e4c2ea98b16ea3ad991d5, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:54.621727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4df536-6750-4a09-adbd-f26ca44edc9b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:54.719487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1336/2714] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:54 2022] PUT /identity/v3/projects/83bd52dbd811457c80f9218ab51a2255/users/341db551fa3e4c2ea98b16ea3ad991d5/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:54.723515 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4df536-6750-4a09-adbd-f26ca44edc9b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:54.782948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0beda7bc43be407696c720ba86a92679'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:54.789620 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4df536-6750-4a09-adbd-f26ca44edc9b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:54.795150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16a8a7-c2b3-44bc-90c7-9343d0b117b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:54.795780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16a8a7-c2b3-44bc-90c7-9343d0b117b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:54.796068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16a8a7-c2b3-44bc-90c7-9343d0b117b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:54.796365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16a8a7-c2b3-44bc-90c7-9343d0b117b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:54.797071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16a8a7-c2b3-44bc-90c7-9343d0b117b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:54.798733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16a8a7-c2b3-44bc-90c7-9343d0b117b3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:54.799266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0beda7bc43be407696c720ba86a92679'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:54.800298 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0beda7bc43be407696c720ba86a92679'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:54.815081 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0beda7bc43be407696c720ba86a92679'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:54.815571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1337/2715] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:36:54.819390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16a8a7-c2b3-44bc-90c7-9343d0b117b3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:54.826625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:54.830622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16a8a7-c2b3-44bc-90c7-9343d0b117b3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:54.837025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe37bc4-904a-44dd-b12a-160b7f1d3e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:54.837726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe37bc4-904a-44dd-b12a-160b7f1d3e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:54.838011 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe37bc4-904a-44dd-b12a-160b7f1d3e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:54.838292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe37bc4-904a-44dd-b12a-160b7f1d3e6f admin admin] PATH_INFO: `/v3/projects/83bd52dbd811457c80f9218ab51a2255/users/341db551fa3e4c2ea98b16ea3ad991d5/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:54.840256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe37bc4-904a-44dd-b12a-160b7f1d3e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=83bd52dbd811457c80f9218ab51a2255, user_id=341db551fa3e4c2ea98b16ea3ad991d5, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:54.842914 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe37bc4-904a-44dd-b12a-160b7f1d3e6f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:54.854034 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1338/2716] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:54 2022] PUT /identity/v3/projects/83bd52dbd811457c80f9218ab51a2255/users/341db551fa3e4c2ea98b16ea3ad991d5/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:54.864252 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cade8e-0725-4ec0-9b10-9351c57c04b8 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:54.864546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cade8e-0725-4ec0-9b10-9351c57c04b8 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:54.864881 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cade8e-0725-4ec0-9b10-9351c57c04b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:54.876838 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0beda7bc43be407696c720ba86a92679'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:54.884874 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1379/2717] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:54 2022] DELETE /identity/v3/users/0beda7bc43be407696c720ba86a92679 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:36:54.896739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b2cade8e-0725-4ec0-9b10-9351c57c04b8 None None] MFA Rules not processed for user `341db551fa3e4c2ea98b16ea3ad991d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:54.897675 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b2cade8e-0725-4ec0-9b10-9351c57c04b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:54.980920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1339/2718] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:36:54.991924 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:54.993698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:55.012926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:55.053398 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0d362-ba87-4e6e-af0d-5680544685fc admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:55.057164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406227d5-aad4-4f8b-a2a2-a6a0a08ad239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:55.057815 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406227d5-aad4-4f8b-a2a2-a6a0a08ad239 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:55.058102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406227d5-aad4-4f8b-a2a2-a6a0a08ad239 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:55.058380 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406227d5-aad4-4f8b-a2a2-a6a0a08ad239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:55.060142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-406227d5-aad4-4f8b-a2a2-a6a0a08ad239 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:55.121438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406227d5-aad4-4f8b-a2a2-a6a0a08ad239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:55.122889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406227d5-aad4-4f8b-a2a2-a6a0a08ad239 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:55.133449 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1380/2719] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:55.423430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cade8e-0725-4ec0-9b10-9351c57c04b8 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:55.424884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b2cade8e-0725-4ec0-9b10-9351c57c04b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:55.513062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cade8e-0725-4ec0-9b10-9351c57c04b8 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:55.517671 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:55.518348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:55.518632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:55.518913 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] PATH_INFO: `/v3/projects/3df753c5cf5543c099bc727131e0d18f` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:55.520093 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df753c5cf5543c099bc727131e0d18f)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:55.522096 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:55.630876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df753c5cf5543c099bc727131e0d18f', 'request_id': 'req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:55.637089 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df753c5cf5543c099bc727131e0d18f', 'request_id': 'req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:55.643625 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df753c5cf5543c099bc727131e0d18f', 'request_id': 'req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:55.738608 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] The token cache is being invalidate because project 3df753c5cf5543c099bc727131e0d18f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:55.739104 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:55.739940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-161130ae-80fd-4034-ba1e-a60dc3cfa1f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:55.742366 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1340/2720] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:55 2022] DELETE /identity/v3/projects/3df753c5cf5543c099bc727131e0d18f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:55.748299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b358615-25bf-43b5-8c09-4251d273bcee None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:55.748794 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b358615-25bf-43b5-8c09-4251d273bcee None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:55.749214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b358615-25bf-43b5-8c09-4251d273bcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:55.769852 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-9b358615-25bf-43b5-8c09-4251d273bcee None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:55.771290 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9b358615-25bf-43b5-8c09-4251d273bcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:55.829880 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1381/2721] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:55.838869 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe774a-fda7-434c-ad8d-ea9f44fb7e03 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:55.840559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe774a-fda7-434c-ad8d-ea9f44fb7e03 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:55.840559 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe774a-fda7-434c-ad8d-ea9f44fb7e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:55.853827 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:55.859940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6ffe774a-fda7-434c-ad8d-ea9f44fb7e03 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:55.861348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6ffe774a-fda7-434c-ad8d-ea9f44fb7e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:55.871278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1341/2722] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:55.876279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b358615-25bf-43b5-8c09-4251d273bcee None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:55.891901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b358615-25bf-43b5-8c09-4251d273bcee None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:55.896570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250830a-82d2-4aa8-9cfd-adb1c08131f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:55.897187 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250830a-82d2-4aa8-9cfd-adb1c08131f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:55.897508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250830a-82d2-4aa8-9cfd-adb1c08131f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:55.897832 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250830a-82d2-4aa8-9cfd-adb1c08131f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:55.898575 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250830a-82d2-4aa8-9cfd-adb1c08131f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:55.898892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250830a-82d2-4aa8-9cfd-adb1c08131f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:55.900543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250830a-82d2-4aa8-9cfd-adb1c08131f7 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:55.913209 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1382/2723] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:36:55.918069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe774a-fda7-434c-ad8d-ea9f44fb7e03 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:55.927775 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe774a-fda7-434c-ad8d-ea9f44fb7e03 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:55.933430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf640-174c-4f0f-a04e-d9a75df6fd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:55.934021 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf640-174c-4f0f-a04e-d9a75df6fd70 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:55.934306 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf640-174c-4f0f-a04e-d9a75df6fd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:55.934578 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf640-174c-4f0f-a04e-d9a75df6fd70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:55.937723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcf640-174c-4f0f-a04e-d9a75df6fd70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:55.937723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcf640-174c-4f0f-a04e-d9a75df6fd70 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:56.056846 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1342/2724] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:55 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:56.060893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250830a-82d2-4aa8-9cfd-adb1c08131f7 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:56.072410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250830a-82d2-4aa8-9cfd-adb1c08131f7 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:56.126417 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=100 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:56.132778 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd38d40-0852-41b6-b4c5-668ec6da1b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:56.133484 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd38d40-0852-41b6-b4c5-668ec6da1b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:56.133783 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd38d40-0852-41b6-b4c5-668ec6da1b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:56.134070 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd38d40-0852-41b6-b4c5-668ec6da1b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:56.134853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd38d40-0852-41b6-b4c5-668ec6da1b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:56.136857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd38d40-0852-41b6-b4c5-668ec6da1b90 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:56.222252 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1383/2725] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:56 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:56.227285 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf640-174c-4f0f-a04e-d9a75df6fd70 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:56.237692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf640-174c-4f0f-a04e-d9a75df6fd70 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:56.242363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb463c-0df2-43be-b4a8-007a8a778173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:56.242975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb463c-0df2-43be-b4a8-007a8a778173 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:56.243892 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb463c-0df2-43be-b4a8-007a8a778173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:56.243892 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb463c-0df2-43be-b4a8-007a8a778173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:56.244218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb463c-0df2-43be-b4a8-007a8a778173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:56.245879 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb463c-0df2-43be-b4a8-007a8a778173 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:56.255895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1343/2726] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:56.260920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd38d40-0852-41b6-b4c5-668ec6da1b90 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:56.271092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd38d40-0852-41b6-b4c5-668ec6da1b90 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:56.276420 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4e61-f23b-45a8-a85e-7c471cf9b949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:56.277033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4e61-f23b-45a8-a85e-7c471cf9b949 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:56.277345 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4e61-f23b-45a8-a85e-7c471cf9b949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:56.277637 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4e61-f23b-45a8-a85e-7c471cf9b949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:56.278340 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b4e61-f23b-45a8-a85e-7c471cf9b949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:56.279947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b4e61-f23b-45a8-a85e-7c471cf9b949 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:56.290843 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1384/2727] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:56.297841 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb463c-0df2-43be-b4a8-007a8a778173 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:56.310643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb463c-0df2-43be-b4a8-007a8a778173 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:56.315533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84612684-a4b7-4eb8-8e2a-279ae61708ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:56.316204 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84612684-a4b7-4eb8-8e2a-279ae61708ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:56.316495 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84612684-a4b7-4eb8-8e2a-279ae61708ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:56.316779 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84612684-a4b7-4eb8-8e2a-279ae61708ed admin admin] PATH_INFO: `/v3/projects/6ff5b43120364c88b03974d710b9377c/users/9c925dd4b54346a287052e5f226625f2/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:56.333215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84612684-a4b7-4eb8-8e2a-279ae61708ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ff5b43120364c88b03974d710b9377c, user_id=9c925dd4b54346a287052e5f226625f2, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:56.336803 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84612684-a4b7-4eb8-8e2a-279ae61708ed admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:56.457544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1344/2728] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:56 2022] PUT /identity/v3/projects/6ff5b43120364c88b03974d710b9377c/users/9c925dd4b54346a287052e5f226625f2/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:56.470647 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb3f76-434c-460e-ba07-59e7d1f1dac2 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:56.470933 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb3f76-434c-460e-ba07-59e7d1f1dac2 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:56.471216 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb3f76-434c-460e-ba07-59e7d1f1dac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:56.500907 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-eafb3f76-434c-460e-ba07-59e7d1f1dac2 None None] MFA Rules not processed for user `9c925dd4b54346a287052e5f226625f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:56.502371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-eafb3f76-434c-460e-ba07-59e7d1f1dac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:56.577546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1385/2729] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:56.582453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84612684-a4b7-4eb8-8e2a-279ae61708ed admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:56.649265 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84612684-a4b7-4eb8-8e2a-279ae61708ed admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:56.653981 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c43ccf-8952-43fb-ba5d-15f82aae4fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:56.654593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c43ccf-8952-43fb-ba5d-15f82aae4fba admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:56.654875 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c43ccf-8952-43fb-ba5d-15f82aae4fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:56.655150 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c43ccf-8952-43fb-ba5d-15f82aae4fba admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:56.655815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c43ccf-8952-43fb-ba5d-15f82aae4fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:56.657743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c43ccf-8952-43fb-ba5d-15f82aae4fba admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:56.747449 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1345/2730] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:56 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:56.748425 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb3f76-434c-460e-ba07-59e7d1f1dac2 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:56.760089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb3f76-434c-460e-ba07-59e7d1f1dac2 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:56.816343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e43555-781a-4548-9028-6d941150db42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:56.816969 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e43555-781a-4548-9028-6d941150db42 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:56.817255 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e43555-781a-4548-9028-6d941150db42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:56.817570 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e43555-781a-4548-9028-6d941150db42 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:56.818287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e43555-781a-4548-9028-6d941150db42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:56.820802 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e43555-781a-4548-9028-6d941150db42 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:56.907601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1386/2731] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:36:56 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:36:56.910502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c43ccf-8952-43fb-ba5d-15f82aae4fba admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:56.921903 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c43ccf-8952-43fb-ba5d-15f82aae4fba admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:56.926818 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7d981-0f34-4902-b3fe-8e5fae1e8540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:56.927503 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d981-0f34-4902-b3fe-8e5fae1e8540 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:56.927796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d981-0f34-4902-b3fe-8e5fae1e8540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:56.928111 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d981-0f34-4902-b3fe-8e5fae1e8540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:56.928812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7d981-0f34-4902-b3fe-8e5fae1e8540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:56.931226 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7d981-0f34-4902-b3fe-8e5fae1e8540 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:56.943648 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1346/2732] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:56.948721 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e43555-781a-4548-9028-6d941150db42 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:56.959224 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e43555-781a-4548-9028-6d941150db42 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:56.964166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003e64e-74cc-4cb3-bdb6-f16293709232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:56.964828 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003e64e-74cc-4cb3-bdb6-f16293709232 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:56.965116 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003e64e-74cc-4cb3-bdb6-f16293709232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:56.965438 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003e64e-74cc-4cb3-bdb6-f16293709232 admin admin] PATH_INFO: `/v3/projects/019fb4ad70b245cfa361f69dbb0fc9c3/users/518c2a98e9854311a67e7717184f95ff/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:56.981036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003e64e-74cc-4cb3-bdb6-f16293709232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=019fb4ad70b245cfa361f69dbb0fc9c3, user_id=518c2a98e9854311a67e7717184f95ff, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:56.983711 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003e64e-74cc-4cb3-bdb6-f16293709232 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:57.071330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1387/2733] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:56 2022] PUT /identity/v3/projects/019fb4ad70b245cfa361f69dbb0fc9c3/users/518c2a98e9854311a67e7717184f95ff/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:57.078369 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7d981-0f34-4902-b3fe-8e5fae1e8540 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:57.088889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7d981-0f34-4902-b3fe-8e5fae1e8540 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:57.149638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7828f5-0879-4739-ac42-48eeb4631e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:57.150268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7828f5-0879-4739-ac42-48eeb4631e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:57.150556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7828f5-0879-4739-ac42-48eeb4631e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:57.150970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7828f5-0879-4739-ac42-48eeb4631e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:57.151674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7828f5-0879-4739-ac42-48eeb4631e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:57.153283 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7828f5-0879-4739-ac42-48eeb4631e78 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:57.164200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1347/2734] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:36:57.169515 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003e64e-74cc-4cb3-bdb6-f16293709232 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:57.180840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003e64e-74cc-4cb3-bdb6-f16293709232 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:57.185786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c516b43-0896-4ac2-9a85-472273ed0770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:57.186450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c516b43-0896-4ac2-9a85-472273ed0770 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:57.186740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c516b43-0896-4ac2-9a85-472273ed0770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:57.187022 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c516b43-0896-4ac2-9a85-472273ed0770 admin admin] PATH_INFO: `/v3/projects/019fb4ad70b245cfa361f69dbb0fc9c3/users/518c2a98e9854311a67e7717184f95ff/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:57.189003 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c516b43-0896-4ac2-9a85-472273ed0770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=019fb4ad70b245cfa361f69dbb0fc9c3, user_id=518c2a98e9854311a67e7717184f95ff, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:57.191719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c516b43-0896-4ac2-9a85-472273ed0770 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:57.202725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1388/2735] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:36:57 2022] PUT /identity/v3/projects/019fb4ad70b245cfa361f69dbb0fc9c3/users/518c2a98e9854311a67e7717184f95ff/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:57.213392 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03628ec2-fd89-4a6b-8890-e12ff4915210 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:57.213752 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03628ec2-fd89-4a6b-8890-e12ff4915210 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:57.215755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03628ec2-fd89-4a6b-8890-e12ff4915210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:57.241267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:57.244953 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-03628ec2-fd89-4a6b-8890-e12ff4915210 None None] MFA Rules not processed for user `518c2a98e9854311a67e7717184f95ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:36:57.246319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-03628ec2-fd89-4a6b-8890-e12ff4915210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:57.317769 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1348/2736] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:36:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:36:57.330353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c516b43-0896-4ac2-9a85-472273ed0770 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:57.332048 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-6c516b43-0896-4ac2-9a85-472273ed0770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:57.355609 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:57.393791 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c516b43-0896-4ac2-9a85-472273ed0770 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:57.397532 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc927b-e10a-42c2-b310-0c18e58e87ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:57.398883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc927b-e10a-42c2-b310-0c18e58e87ad service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:57.399247 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc927b-e10a-42c2-b310-0c18e58e87ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:57.399549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc927b-e10a-42c2-b310-0c18e58e87ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:57.410428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc927b-e10a-42c2-b310-0c18e58e87ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:57.411900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc927b-e10a-42c2-b310-0c18e58e87ad service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:57.423472 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1389/2737] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:36:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:36:58.549568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03628ec2-fd89-4a6b-8890-e12ff4915210 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:58.559519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03628ec2-fd89-4a6b-8890-e12ff4915210 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:58.564397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c7d9b-3292-4015-9a3a-e0eb05c027ef tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '518c2a98e9854311a67e7717184f95ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '019fb4ad70b245cfa361f69dbb0fc9c3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:58.565091 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c7d9b-3292-4015-9a3a-e0eb05c027ef tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:58.565412 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c7d9b-3292-4015-9a3a-e0eb05c027ef tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:58.565707 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c7d9b-3292-4015-9a3a-e0eb05c027ef tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:58.566463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c7d9b-3292-4015-9a3a-e0eb05c027ef tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:58.567006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c7d9b-3292-4015-9a3a-e0eb05c027ef tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:58.568728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c7d9b-3292-4015-9a3a-e0eb05c027ef tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:58.581925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1349/2738] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:36:58.587412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc927b-e10a-42c2-b310-0c18e58e87ad service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:58.597041 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc927b-e10a-42c2-b310-0c18e58e87ad service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:58.602163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e9bad-62cb-4a56-863f-0595c22972c9 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '518c2a98e9854311a67e7717184f95ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '019fb4ad70b245cfa361f69dbb0fc9c3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:58.602777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e9bad-62cb-4a56-863f-0595c22972c9 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:58.603064 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e9bad-62cb-4a56-863f-0595c22972c9 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:58.603348 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e9bad-62cb-4a56-863f-0595c22972c9 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:58.604035 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e9bad-62cb-4a56-863f-0595c22972c9 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:58.606125 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e9bad-62cb-4a56-863f-0595c22972c9 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:58.724333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1390/2739] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:36:58 2022] POST /identity/v3/projects => generated 355 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:36:58.728543 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c7d9b-3292-4015-9a3a-e0eb05c027ef tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:58.738614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c7d9b-3292-4015-9a3a-e0eb05c027ef tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:58.799847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ba47c-0a4a-46d5-864a-0d726b9979bc tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '518c2a98e9854311a67e7717184f95ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '019fb4ad70b245cfa361f69dbb0fc9c3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:58.800397 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ba47c-0a4a-46d5-864a-0d726b9979bc tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:58.800684 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ba47c-0a4a-46d5-864a-0d726b9979bc tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:58.801000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ba47c-0a4a-46d5-864a-0d726b9979bc tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:58.801871 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ba47c-0a4a-46d5-864a-0d726b9979bc tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:36:58.802187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ba47c-0a4a-46d5-864a-0d726b9979bc tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:58.803844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ba47c-0a4a-46d5-864a-0d726b9979bc tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:58.816932 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1350/2740] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:36:58.857826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e9bad-62cb-4a56-863f-0595c22972c9 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:58.869329 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e9bad-62cb-4a56-863f-0595c22972c9 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:58.874015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '518c2a98e9854311a67e7717184f95ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '019fb4ad70b245cfa361f69dbb0fc9c3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:58.874693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:58.874982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:58.875266 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] PATH_INFO: `/v3/projects/382cc474eebb4c15955218eec7a63eb4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:58.876456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: Authorizing `identity:delete_project(project_id=382cc474eebb4c15955218eec7a63eb4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:58.878428 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:59.013494 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '382cc474eebb4c15955218eec7a63eb4', 'request_id': 'req-39f4fd09-2f8a-4006-aa4c-850905572b57'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:59.018458 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '382cc474eebb4c15955218eec7a63eb4', 'request_id': 'req-39f4fd09-2f8a-4006-aa4c-850905572b57'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:59.082561 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '382cc474eebb4c15955218eec7a63eb4', 'request_id': 'req-39f4fd09-2f8a-4006-aa4c-850905572b57'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:59.097142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] The token cache is being invalidate because project 382cc474eebb4c15955218eec7a63eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:36:59.098644 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:59.098644 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:36:59.099640 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1391/2741] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:36:58 2022] DELETE /identity/v3/projects/382cc474eebb4c15955218eec7a63eb4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:36:59.689896 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ba47c-0a4a-46d5-864a-0d726b9979bc tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:59.691760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-788ba47c-0a4a-46d5-864a-0d726b9979bc tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:59.758539 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ba47c-0a4a-46d5-864a-0d726b9979bc tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:59.784281 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:36:59.803293 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:36:59.805487 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:36:59.819956 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:59.820773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:59.821056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:59.821435 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] PATH_INFO: `/v3/users/ba82cc46e4fe4d2c8a39a6d23e03145e` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:59.822708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba82cc46e4fe4d2c8a39a6d23e03145e)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:59.824694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:59.872771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4fd09-2f8a-4006-aa4c-850905572b57 tempest-VolumeQuotaClassesTest-374071831 tempest-VolumeQuotaClassesTest-374071831-project-admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:36:59.877237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:36:59.877897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:36:59.878181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:36:59.878475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] PATH_INFO: `/v3/users/9c925dd4b54346a287052e5f226625f2` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:36:59.879618 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c925dd4b54346a287052e5f226625f2)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:36:59.881660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:36:59.906473 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:00.034233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba82cc46e4fe4d2c8a39a6d23e03145e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:00.046350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba82cc46e4fe4d2c8a39a6d23e03145e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:00.053268 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba82cc46e4fe4d2c8a39a6d23e03145e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:00.079225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c925dd4b54346a287052e5f226625f2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:00.093037 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c925dd4b54346a287052e5f226625f2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:00.093804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c925dd4b54346a287052e5f226625f2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:00.100689 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c925dd4b54346a287052e5f226625f2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:00.134502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba82cc46e4fe4d2c8a39a6d23e03145e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:00.135168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5adf73d1-f1f4-4aa0-abf3-8ab4500f22dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba82cc46e4fe4d2c8a39a6d23e03145e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:00.142604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1351/2742] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:59 2022] DELETE /identity/v3/users/ba82cc46e4fe4d2c8a39a6d23e03145e => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:00.147148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee847f-c2f6-40d9-bb7d-fc806c3dbe1e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:00.147440 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee847f-c2f6-40d9-bb7d-fc806c3dbe1e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:00.148426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee847f-c2f6-40d9-bb7d-fc806c3dbe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:00.168256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-dfee847f-c2f6-40d9-bb7d-fc806c3dbe1e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:00.169496 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dfee847f-c2f6-40d9-bb7d-fc806c3dbe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:00.182149 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-705947e7-3af4-40e2-b764-d1113359028d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c925dd4b54346a287052e5f226625f2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:00.189701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1392/2743] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:36:59 2022] DELETE /identity/v3/users/9c925dd4b54346a287052e5f226625f2 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:00.195770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829cb0e-de8c-40aa-8559-25e13d22e12f None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:00.196220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829cb0e-de8c-40aa-8559-25e13d22e12f None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:00.196709 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829cb0e-de8c-40aa-8559-25e13d22e12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:00.217991 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f829cb0e-de8c-40aa-8559-25e13d22e12f None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:00.219261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f829cb0e-de8c-40aa-8559-25e13d22e12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:00.230376 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1352/2744] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:00.240234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee847f-c2f6-40d9-bb7d-fc806c3dbe1e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:00.242040 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dfee847f-c2f6-40d9-bb7d-fc806c3dbe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:00.287934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1393/2745] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:00.298402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829cb0e-de8c-40aa-8559-25e13d22e12f None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:00.302016 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:00.307571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee847f-c2f6-40d9-bb7d-fc806c3dbe1e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:00.360754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c42d3-b201-4c51-a9c3-64d79dd9ae67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:00.361446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c42d3-b201-4c51-a9c3-64d79dd9ae67 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:00.361902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c42d3-b201-4c51-a9c3-64d79dd9ae67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:00.362186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c42d3-b201-4c51-a9c3-64d79dd9ae67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:00.367410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829cb0e-de8c-40aa-8559-25e13d22e12f None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:00.373513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704b11c-26f0-46e2-8b98-421f0a68029d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:00.374179 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704b11c-26f0-46e2-8b98-421f0a68029d service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:00.374468 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704b11c-26f0-46e2-8b98-421f0a68029d service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:00.374786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704b11c-26f0-46e2-8b98-421f0a68029d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:00.379815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c42d3-b201-4c51-a9c3-64d79dd9ae67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:00.381507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c42d3-b201-4c51-a9c3-64d79dd9ae67 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:00.385588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704b11c-26f0-46e2-8b98-421f0a68029d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:00.386987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704b11c-26f0-46e2-8b98-421f0a68029d service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:00.394670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1353/2746] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:00.398432 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1394/2747] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:00.712617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c42d3-b201-4c51-a9c3-64d79dd9ae67 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:00.714548 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b35c42d3-b201-4c51-a9c3-64d79dd9ae67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:00.779114 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c42d3-b201-4c51-a9c3-64d79dd9ae67 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:00.782996 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ec8cb-3c88-4700-b2e3-ba30bc47d4a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:00.783629 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ec8cb-3c88-4700-b2e3-ba30bc47d4a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:00.783918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ec8cb-3c88-4700-b2e3-ba30bc47d4a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:00.784274 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ec8cb-3c88-4700-b2e3-ba30bc47d4a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:00.786063 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-953ec8cb-3c88-4700-b2e3-ba30bc47d4a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:00.852919 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ec8cb-3c88-4700-b2e3-ba30bc47d4a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:00.854704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ec8cb-3c88-4700-b2e3-ba30bc47d4a4 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:00.864726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1354/2748] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:00.948582 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704b11c-26f0-46e2-8b98-421f0a68029d service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:00.950120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a704b11c-26f0-46e2-8b98-421f0a68029d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:00.965225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ec8cb-3c88-4700-b2e3-ba30bc47d4a4 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:00.966929 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-953ec8cb-3c88-4700-b2e3-ba30bc47d4a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:00.970043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704b11c-26f0-46e2-8b98-421f0a68029d service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:00.981262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:00.981905 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:00.982202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:00.982451 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ec8cb-3c88-4700-b2e3-ba30bc47d4a4 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:00.982577 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] PATH_INFO: `/v3/projects/3f8d30bf32cf4aab8ca1cfbce133de80` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:00.985261 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f8d30bf32cf4aab8ca1cfbce133de80)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:00.988905 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:00.989823 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:00.989947 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:00.990767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:00.991211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] PATH_INFO: `/v3/projects/6ff5b43120364c88b03974d710b9377c` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:00.992933 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ff5b43120364c88b03974d710b9377c)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:00.995954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:01.206920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:01.214264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f8d30bf32cf4aab8ca1cfbce133de80', 'request_id': 'req-3ceafd95-39f1-4374-83d0-af4fe47391d2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.215049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ff5b43120364c88b03974d710b9377c', 'request_id': 'req-697ab822-d685-49b9-bfeb-a211deea6949'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.216367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:01.220034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f8d30bf32cf4aab8ca1cfbce133de80', 'request_id': 'req-3ceafd95-39f1-4374-83d0-af4fe47391d2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.221178 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ff5b43120364c88b03974d710b9377c', 'request_id': 'req-697ab822-d685-49b9-bfeb-a211deea6949'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.227836 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ff5b43120364c88b03974d710b9377c', 'request_id': 'req-697ab822-d685-49b9-bfeb-a211deea6949'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.306306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f8d30bf32cf4aab8ca1cfbce133de80', 'request_id': 'req-3ceafd95-39f1-4374-83d0-af4fe47391d2'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.317981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] The token cache is being invalidate because project 3f8d30bf32cf4aab8ca1cfbce133de80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:01.318279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.318927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ceafd95-39f1-4374-83d0-af4fe47391d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.321493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1395/2749] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:00 2022] DELETE /identity/v3/projects/3f8d30bf32cf4aab8ca1cfbce133de80 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:01.326304 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cadea-ff37-4796-8dbb-d8a883bfecef None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.326599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cadea-ff37-4796-8dbb-d8a883bfecef None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.326890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cadea-ff37-4796-8dbb-d8a883bfecef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.346356 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-514cadea-ff37-4796-8dbb-d8a883bfecef None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:01.347583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-514cadea-ff37-4796-8dbb-d8a883bfecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:01.408621 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1396/2750] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:01.432278 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] The token cache is being invalidate because project 6ff5b43120364c88b03974d710b9377c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:01.432690 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.433514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-697ab822-d685-49b9-bfeb-a211deea6949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.435216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1355/2751] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:00 2022] DELETE /identity/v3/projects/6ff5b43120364c88b03974d710b9377c => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:01.439550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cadea-ff37-4796-8dbb-d8a883bfecef None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:01.441109 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-514cadea-ff37-4796-8dbb-d8a883bfecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:01.455328 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cadea-ff37-4796-8dbb-d8a883bfecef None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:01.460752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:01.461463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.461754 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.462034 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] PATH_INFO: `/v3/users/518c2a98e9854311a67e7717184f95ff` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.463344 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=518c2a98e9854311a67e7717184f95ff)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:01.465405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:01.480364 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdd576-fdde-4eb9-bf2c-4130fa32b7c9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.480699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdd576-fdde-4eb9-bf2c-4130fa32b7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.480942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdd576-fdde-4eb9-bf2c-4130fa32b7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.503313 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5fcdd576-fdde-4eb9-bf2c-4130fa32b7c9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:01.504569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5fcdd576-fdde-4eb9-bf2c-4130fa32b7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:01.513256 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1356/2752] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:01.517794 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdd576-fdde-4eb9-bf2c-4130fa32b7c9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:01.526360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdd576-fdde-4eb9-bf2c-4130fa32b7c9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:01.530963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc4425-aac7-49e1-9514-7185ac39147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:01.531563 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc4425-aac7-49e1-9514-7185ac39147b admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.531847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc4425-aac7-49e1-9514-7185ac39147b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.532126 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc4425-aac7-49e1-9514-7185ac39147b admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.532820 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc4425-aac7-49e1-9514-7185ac39147b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:01.533158 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc4425-aac7-49e1-9514-7185ac39147b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:01.534794 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc4425-aac7-49e1-9514-7185ac39147b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:01.547060 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1357/2753] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:37:01.551401 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc4425-aac7-49e1-9514-7185ac39147b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:01.561730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc4425-aac7-49e1-9514-7185ac39147b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:01.566609 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17424224-d7b6-4e68-95b7-097833d6c06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:01.566931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17424224-d7b6-4e68-95b7-097833d6c06b admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.567218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17424224-d7b6-4e68-95b7-097833d6c06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.567504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17424224-d7b6-4e68-95b7-097833d6c06b admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.568175 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17424224-d7b6-4e68-95b7-097833d6c06b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:01.570330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17424224-d7b6-4e68-95b7-097833d6c06b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:01.685277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1358/2754] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:01 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:01.689994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17424224-d7b6-4e68-95b7-097833d6c06b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:01.696146 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '518c2a98e9854311a67e7717184f95ff'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.714717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '518c2a98e9854311a67e7717184f95ff'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.718386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '518c2a98e9854311a67e7717184f95ff'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.723023 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '518c2a98e9854311a67e7717184f95ff'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.760326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17424224-d7b6-4e68-95b7-097833d6c06b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:01.765872 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f433b58-1c54-4459-9b66-ed39788e427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:01.766501 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f433b58-1c54-4459-9b66-ed39788e427c admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.766790 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f433b58-1c54-4459-9b66-ed39788e427c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.767068 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f433b58-1c54-4459-9b66-ed39788e427c admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.767742 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f433b58-1c54-4459-9b66-ed39788e427c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:01.769809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f433b58-1c54-4459-9b66-ed39788e427c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:01.791063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '518c2a98e9854311a67e7717184f95ff'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:01.797731 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1397/2755] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:01 2022] DELETE /identity/v3/users/518c2a98e9854311a67e7717184f95ff => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:01.887560 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1359/2756] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:01 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:01.888856 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:01.890590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:01.937480 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:01.958860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41801af-c897-476a-9869-9714f86decc8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:01.964446 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc3b5c-b326-461a-b605-350dc556ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:01.965082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc3b5c-b326-461a-b605-350dc556ba3f admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.965405 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc3b5c-b326-461a-b605-350dc556ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.965690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc3b5c-b326-461a-b605-350dc556ba3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.966415 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc3b5c-b326-461a-b605-350dc556ba3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:01.968032 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc3b5c-b326-461a-b605-350dc556ba3f admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:01.978755 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1398/2757] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:01.982974 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f433b58-1c54-4459-9b66-ed39788e427c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:01.993488 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f433b58-1c54-4459-9b66-ed39788e427c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:01.998899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998371a7-612e-408f-8944-acbdd4145e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:01.999682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998371a7-612e-408f-8944-acbdd4145e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.999967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998371a7-612e-408f-8944-acbdd4145e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.000241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998371a7-612e-408f-8944-acbdd4145e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.000970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998371a7-612e-408f-8944-acbdd4145e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.002665 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998371a7-612e-408f-8944-acbdd4145e27 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.013691 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1360/2758] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:02.018077 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc3b5c-b326-461a-b605-350dc556ba3f admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.029238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc3b5c-b326-461a-b605-350dc556ba3f admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.034590 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf15b6-6c4c-43c8-91ee-24e957fa8bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.035112 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf15b6-6c4c-43c8-91ee-24e957fa8bec admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.035402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf15b6-6c4c-43c8-91ee-24e957fa8bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.035707 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf15b6-6c4c-43c8-91ee-24e957fa8bec admin admin] PATH_INFO: `/v3/projects/3eede0dda6924c4e979c7156520cd457/users/9d27158da27b43f5b6bda86ef68a76ef/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.060751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf15b6-6c4c-43c8-91ee-24e957fa8bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eede0dda6924c4e979c7156520cd457, user_id=9d27158da27b43f5b6bda86ef68a76ef, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.062120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf15b6-6c4c-43c8-91ee-24e957fa8bec admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.151867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1399/2759] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:02 2022] PUT /identity/v3/projects/3eede0dda6924c4e979c7156520cd457/users/9d27158da27b43f5b6bda86ef68a76ef/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:02.162771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb83bb3-54c6-4270-af3e-d7fb09ca7a82 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.163232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb83bb3-54c6-4270-af3e-d7fb09ca7a82 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.164452 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb83bb3-54c6-4270-af3e-d7fb09ca7a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.196216 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-7cb83bb3-54c6-4270-af3e-d7fb09ca7a82 None None] MFA Rules not processed for user `9d27158da27b43f5b6bda86ef68a76ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.197525 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7cb83bb3-54c6-4270-af3e-d7fb09ca7a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.263681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1361/2760] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.270272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf15b6-6c4c-43c8-91ee-24e957fa8bec admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.270272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8edf15b6-6c4c-43c8-91ee-24e957fa8bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.284087 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb83bb3-54c6-4270-af3e-d7fb09ca7a82 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.285618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7cb83bb3-54c6-4270-af3e-d7fb09ca7a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.293254 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:02.342881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf15b6-6c4c-43c8-91ee-24e957fa8bec admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.346765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb83bb3-54c6-4270-af3e-d7fb09ca7a82 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.352882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.353246 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f742dc-a20d-4598-ad8b-056c01cdb7e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4fe04b11ec412e8f4eb67c6e54b8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.353740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.354506 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.354640 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f742dc-a20d-4598-ad8b-056c01cdb7e4 service glance] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.354743 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] PATH_INFO: `/v3/projects/019fb4ad70b245cfa361f69dbb0fc9c3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.354840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f742dc-a20d-4598-ad8b-056c01cdb7e4 service glance] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.354840 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f742dc-a20d-4598-ad8b-056c01cdb7e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.355883 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] RBAC: Authorizing `identity:delete_project(project_id=019fb4ad70b245cfa361f69dbb0fc9c3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.358330 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.364304 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f742dc-a20d-4598-ad8b-056c01cdb7e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.365810 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f742dc-a20d-4598-ad8b-056c01cdb7e4 service glance] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.375699 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1362/2761] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:02.474583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '019fb4ad70b245cfa361f69dbb0fc9c3', 'request_id': 'req-248c2369-3fd0-4fc0-be57-dc14f91b301a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:02.484341 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '019fb4ad70b245cfa361f69dbb0fc9c3', 'request_id': 'req-248c2369-3fd0-4fc0-be57-dc14f91b301a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:02.576024 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '019fb4ad70b245cfa361f69dbb0fc9c3', 'request_id': 'req-248c2369-3fd0-4fc0-be57-dc14f91b301a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:02.588981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] The token cache is being invalidate because project 019fb4ad70b245cfa361f69dbb0fc9c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:02.595736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:02.595736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-248c2369-3fd0-4fc0-be57-dc14f91b301a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:02.595736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1400/2762] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:02 2022] DELETE /identity/v3/projects/019fb4ad70b245cfa361f69dbb0fc9c3 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:02.597882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a5b3e-e6c9-457a-84f0-1b8ba067aec9 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.598171 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a5b3e-e6c9-457a-84f0-1b8ba067aec9 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.598454 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a5b3e-e6c9-457a-84f0-1b8ba067aec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.618486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-274a5b3e-e6c9-457a-84f0-1b8ba067aec9 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.622025 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-274a5b3e-e6c9-457a-84f0-1b8ba067aec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.684743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1363/2763] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.692620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544c077-9e53-4296-b596-aaf976ccb5b3 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.692982 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544c077-9e53-4296-b596-aaf976ccb5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.693354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544c077-9e53-4296-b596-aaf976ccb5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.714922 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e544c077-9e53-4296-b596-aaf976ccb5b3 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.716135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e544c077-9e53-4296-b596-aaf976ccb5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.724127 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1401/2764] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.731847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a5b3e-e6c9-457a-84f0-1b8ba067aec9 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.742730 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a5b3e-e6c9-457a-84f0-1b8ba067aec9 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.749895 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55858f-027d-4a3e-b4ed-c9e956081ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.752350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55858f-027d-4a3e-b4ed-c9e956081ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.752350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55858f-027d-4a3e-b4ed-c9e956081ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.752350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55858f-027d-4a3e-b4ed-c9e956081ab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.752350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55858f-027d-4a3e-b4ed-c9e956081ab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:02.752350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55858f-027d-4a3e-b4ed-c9e956081ab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.754081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55858f-027d-4a3e-b4ed-c9e956081ab3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.767646 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1364/2765] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:02.772182 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544c077-9e53-4296-b596-aaf976ccb5b3 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.781840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544c077-9e53-4296-b596-aaf976ccb5b3 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.786849 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d92201-8c1c-420f-a15d-70afdea5b58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.787397 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d92201-8c1c-420f-a15d-70afdea5b58b admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.787676 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d92201-8c1c-420f-a15d-70afdea5b58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.787948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d92201-8c1c-420f-a15d-70afdea5b58b admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.788656 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d92201-8c1c-420f-a15d-70afdea5b58b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.790620 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d92201-8c1c-420f-a15d-70afdea5b58b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.909381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1402/2766] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:02.914491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55858f-027d-4a3e-b4ed-c9e956081ab3 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.924589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55858f-027d-4a3e-b4ed-c9e956081ab3 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.983148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fff6e-b4e9-40ab-aeed-c7afbc8172d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.983774 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fff6e-b4e9-40ab-aeed-c7afbc8172d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.984061 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fff6e-b4e9-40ab-aeed-c7afbc8172d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.984343 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fff6e-b4e9-40ab-aeed-c7afbc8172d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.985022 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fff6e-b4e9-40ab-aeed-c7afbc8172d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.986989 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fff6e-b4e9-40ab-aeed-c7afbc8172d6 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.113720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1365/2767] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:03.117944 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d92201-8c1c-420f-a15d-70afdea5b58b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.128556 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d92201-8c1c-420f-a15d-70afdea5b58b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.133800 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d189cd6-5ded-4235-b776-3bbbd75a595d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.134412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d189cd6-5ded-4235-b776-3bbbd75a595d admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.134700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d189cd6-5ded-4235-b776-3bbbd75a595d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.134987 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d189cd6-5ded-4235-b776-3bbbd75a595d admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.135698 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d189cd6-5ded-4235-b776-3bbbd75a595d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.137334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d189cd6-5ded-4235-b776-3bbbd75a595d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.148563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1403/2768] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.152462 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fff6e-b4e9-40ab-aeed-c7afbc8172d6 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.169682 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fff6e-b4e9-40ab-aeed-c7afbc8172d6 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.174842 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042002c-d572-4370-8445-b839cb16dbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.175465 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042002c-d572-4370-8445-b839cb16dbbf admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.175754 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042002c-d572-4370-8445-b839cb16dbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.176052 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042002c-d572-4370-8445-b839cb16dbbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.176755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042002c-d572-4370-8445-b839cb16dbbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.178470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042002c-d572-4370-8445-b839cb16dbbf admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.198405 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1366/2769] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.202799 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d189cd6-5ded-4235-b776-3bbbd75a595d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.208325 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:03.214396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d189cd6-5ded-4235-b776-3bbbd75a595d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.219160 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b78d3f-67ac-4477-ab07-cd49cc3165ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.219831 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b78d3f-67ac-4477-ab07-cd49cc3165ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.220122 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b78d3f-67ac-4477-ab07-cd49cc3165ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.220407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b78d3f-67ac-4477-ab07-cd49cc3165ee admin admin] PATH_INFO: `/v3/projects/63ed9e3bd9534d9b8ea769e2ff126343/users/6befb02914f7421d994e9471e4c18f8c/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.241984 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b78d3f-67ac-4477-ab07-cd49cc3165ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=63ed9e3bd9534d9b8ea769e2ff126343, user_id=6befb02914f7421d994e9471e4c18f8c, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.244867 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b78d3f-67ac-4477-ab07-cd49cc3165ee admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.365498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1404/2770] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:03 2022] PUT /identity/v3/projects/63ed9e3bd9534d9b8ea769e2ff126343/users/6befb02914f7421d994e9471e4c18f8c/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:03.437102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63b5d5-15ec-4378-9e54-71721bdab5b5 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.437499 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63b5d5-15ec-4378-9e54-71721bdab5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.437715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63b5d5-15ec-4378-9e54-71721bdab5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.463925 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:03.466920 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-8f63b5d5-15ec-4378-9e54-71721bdab5b5 None None] MFA Rules not processed for user `6befb02914f7421d994e9471e4c18f8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:03.468154 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-8f63b5d5-15ec-4378-9e54-71721bdab5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:03.536141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1367/2771] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:03.548201 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b78d3f-67ac-4477-ab07-cd49cc3165ee admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.549996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-a2b78d3f-67ac-4477-ab07-cd49cc3165ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:03.610178 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b78d3f-67ac-4477-ab07-cd49cc3165ee admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.613827 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28899a-4793-4507-9b08-c48900404710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.614447 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28899a-4793-4507-9b08-c48900404710 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.614740 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28899a-4793-4507-9b08-c48900404710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.615019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28899a-4793-4507-9b08-c48900404710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.623875 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28899a-4793-4507-9b08-c48900404710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.625331 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28899a-4793-4507-9b08-c48900404710 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.635004 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1405/2772] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:03.654342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63b5d5-15ec-4378-9e54-71721bdab5b5 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.720833 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63b5d5-15ec-4378-9e54-71721bdab5b5 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.726611 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.727419 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.727696 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.727979 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] PATH_INFO: `/v3/users/6befb02914f7421d994e9471e4c18f8c` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.729348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6befb02914f7421d994e9471e4c18f8c)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.731291 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.919784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6befb02914f7421d994e9471e4c18f8c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:03.936128 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6befb02914f7421d994e9471e4c18f8c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:03.944007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6befb02914f7421d994e9471e4c18f8c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:04.008622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6befb02914f7421d994e9471e4c18f8c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:04.009509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6befb02914f7421d994e9471e4c18f8c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:04.017067 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1368/2773] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:03 2022] DELETE /identity/v3/users/6befb02914f7421d994e9471e4c18f8c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:04.021718 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd13bf4-9036-40e4-b63a-2ee242decb35 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.022013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd13bf4-9036-40e4-b63a-2ee242decb35 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.022297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd13bf4-9036-40e4-b63a-2ee242decb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.041869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-2bd13bf4-9036-40e4-b63a-2ee242decb35 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:04.043124 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2bd13bf4-9036-40e4-b63a-2ee242decb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:04.104283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1406/2774] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:04.114982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.116931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:04.147007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:04.179740 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2a1fc-d2db-41db-a19e-07a3995044cc admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.183714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5aacc-fd5a-465e-81f5-b30af1431e1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.184551 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5aacc-fd5a-465e-81f5-b30af1431e1e service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.185015 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5aacc-fd5a-465e-81f5-b30af1431e1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.185712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5aacc-fd5a-465e-81f5-b30af1431e1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.196373 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5aacc-fd5a-465e-81f5-b30af1431e1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.197935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5aacc-fd5a-465e-81f5-b30af1431e1e service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.208415 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1369/2775] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:04.738457 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd13bf4-9036-40e4-b63a-2ee242decb35 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.740099 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2bd13bf4-9036-40e4-b63a-2ee242decb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:04.759600 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd13bf4-9036-40e4-b63a-2ee242decb35 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.765517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.766339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.766511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.766857 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] PATH_INFO: `/v3/projects/63ed9e3bd9534d9b8ea769e2ff126343` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.768487 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=63ed9e3bd9534d9b8ea769e2ff126343)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.770470 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.908771 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63ed9e3bd9534d9b8ea769e2ff126343', 'request_id': 'req-aae505a2-4cd3-4048-9cd6-a3ac751750fe'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:04.915018 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63ed9e3bd9534d9b8ea769e2ff126343', 'request_id': 'req-aae505a2-4cd3-4048-9cd6-a3ac751750fe'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:05.028126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63ed9e3bd9534d9b8ea769e2ff126343', 'request_id': 'req-aae505a2-4cd3-4048-9cd6-a3ac751750fe'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:05.041214 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] The token cache is being invalidate because project 63ed9e3bd9534d9b8ea769e2ff126343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:05.041555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:05.042238 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-aae505a2-4cd3-4048-9cd6-a3ac751750fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:05.043981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1407/2776] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:04 2022] DELETE /identity/v3/projects/63ed9e3bd9534d9b8ea769e2ff126343 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:05.048966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2b21-b9b6-4597-b8d4-296ba1653418 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.049491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2b21-b9b6-4597-b8d4-296ba1653418 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.049722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2b21-b9b6-4597-b8d4-296ba1653418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.070650 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-fa9a2b21-b9b6-4597-b8d4-296ba1653418 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:05.071893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-fa9a2b21-b9b6-4597-b8d4-296ba1653418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:05.135211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1370/2777] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:05.142666 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c015ef-f978-4125-8f7c-d22d27735a4d None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.142967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c015ef-f978-4125-8f7c-d22d27735a4d None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.143254 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c015ef-f978-4125-8f7c-d22d27735a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.153147 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:05.165616 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f6c015ef-f978-4125-8f7c-d22d27735a4d None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:05.166878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f6c015ef-f978-4125-8f7c-d22d27735a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:05.175015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1408/2778] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:05.179299 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a2b21-b9b6-4597-b8d4-296ba1653418 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.190186 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a2b21-b9b6-4597-b8d4-296ba1653418 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.195054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6357db-1bcd-437a-927f-2f3f72d640fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.196277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6357db-1bcd-437a-927f-2f3f72d640fb admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.196587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6357db-1bcd-437a-927f-2f3f72d640fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.196865 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6357db-1bcd-437a-927f-2f3f72d640fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.197592 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6357db-1bcd-437a-927f-2f3f72d640fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:05.197901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6357db-1bcd-437a-927f-2f3f72d640fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.199761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6357db-1bcd-437a-927f-2f3f72d640fb admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.212586 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1371/2779] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:05.216816 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c015ef-f978-4125-8f7c-d22d27735a4d None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.229960 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c015ef-f978-4125-8f7c-d22d27735a4d None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.235050 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa37b4-6fdd-482c-ae70-1b0099695d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.235751 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa37b4-6fdd-482c-ae70-1b0099695d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.236040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa37b4-6fdd-482c-ae70-1b0099695d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.236324 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa37b4-6fdd-482c-ae70-1b0099695d01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.237062 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa37b4-6fdd-482c-ae70-1b0099695d01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.239120 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa37b4-6fdd-482c-ae70-1b0099695d01 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.347163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1409/2780] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:05 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:05.347313 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6357db-1bcd-437a-927f-2f3f72d640fb admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.350042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6357db-1bcd-437a-927f-2f3f72d640fb admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.414968 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5ac13-62eb-4336-8e57-269a583deb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.415597 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5ac13-62eb-4336-8e57-269a583deb75 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.415882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5ac13-62eb-4336-8e57-269a583deb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.416161 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5ac13-62eb-4336-8e57-269a583deb75 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.416825 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5ac13-62eb-4336-8e57-269a583deb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.418861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5ac13-62eb-4336-8e57-269a583deb75 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.536959 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1372/2781] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:05 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:05.540893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa37b4-6fdd-482c-ae70-1b0099695d01 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.553274 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa37b4-6fdd-482c-ae70-1b0099695d01 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.558083 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98d67d-95bc-41db-b06c-24638fe63252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.558717 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98d67d-95bc-41db-b06c-24638fe63252 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.559010 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98d67d-95bc-41db-b06c-24638fe63252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.559292 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98d67d-95bc-41db-b06c-24638fe63252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.560015 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98d67d-95bc-41db-b06c-24638fe63252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.561989 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98d67d-95bc-41db-b06c-24638fe63252 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.573045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1410/2782] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:05.577076 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5ac13-62eb-4336-8e57-269a583deb75 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.589474 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5ac13-62eb-4336-8e57-269a583deb75 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.594145 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192053f8-1d96-470e-a2ab-ed2bc9eddd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.594766 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192053f8-1d96-470e-a2ab-ed2bc9eddd35 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.595051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192053f8-1d96-470e-a2ab-ed2bc9eddd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.595334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192053f8-1d96-470e-a2ab-ed2bc9eddd35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.596036 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192053f8-1d96-470e-a2ab-ed2bc9eddd35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.597720 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192053f8-1d96-470e-a2ab-ed2bc9eddd35 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.614931 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1373/2783] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:05.620409 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98d67d-95bc-41db-b06c-24638fe63252 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.631792 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98d67d-95bc-41db-b06c-24638fe63252 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.638768 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0309b0-2148-4e7d-a6b4-0438deb9bd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.639467 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0309b0-2148-4e7d-a6b4-0438deb9bd61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.640789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0309b0-2148-4e7d-a6b4-0438deb9bd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.640789 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0309b0-2148-4e7d-a6b4-0438deb9bd61 admin admin] PATH_INFO: `/v3/projects/c367d92e315848ecb75564e84edf57b3/users/1f3729c8935041aabcd9b123ee46156b/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.662191 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0309b0-2148-4e7d-a6b4-0438deb9bd61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c367d92e315848ecb75564e84edf57b3, user_id=1f3729c8935041aabcd9b123ee46156b, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.664945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0309b0-2148-4e7d-a6b4-0438deb9bd61 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.737906 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1411/2784] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:05 2022] PUT /identity/v3/projects/c367d92e315848ecb75564e84edf57b3/users/1f3729c8935041aabcd9b123ee46156b/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:05.746151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd871c51-a434-47c6-ba58-a634091894bf None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.746448 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd871c51-a434-47c6-ba58-a634091894bf None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.746734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd871c51-a434-47c6-ba58-a634091894bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.776798 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-dd871c51-a434-47c6-ba58-a634091894bf None None] MFA Rules not processed for user `1f3729c8935041aabcd9b123ee46156b`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:05.778075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-dd871c51-a434-47c6-ba58-a634091894bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:05.798210 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:05.855849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1374/2785] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:05.857092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0309b0-2148-4e7d-a6b4-0438deb9bd61 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.925295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0309b0-2148-4e7d-a6b4-0438deb9bd61 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.931183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004494df-3ed8-4e0f-abff-31cefa5a10e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.931519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004494df-3ed8-4e0f-abff-31cefa5a10e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.931854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004494df-3ed8-4e0f-abff-31cefa5a10e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.932135 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004494df-3ed8-4e0f-abff-31cefa5a10e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.932888 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004494df-3ed8-4e0f-abff-31cefa5a10e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.934954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004494df-3ed8-4e0f-abff-31cefa5a10e7 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.047005 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1412/2786] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:05 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:06.051269 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd871c51-a434-47c6-ba58-a634091894bf None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.063470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd871c51-a434-47c6-ba58-a634091894bf None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.126830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570546e1-a60c-42b2-919b-be79e4f903b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.127374 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570546e1-a60c-42b2-919b-be79e4f903b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.127663 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570546e1-a60c-42b2-919b-be79e4f903b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.127942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570546e1-a60c-42b2-919b-be79e4f903b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.128618 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570546e1-a60c-42b2-919b-be79e4f903b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.130614 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570546e1-a60c-42b2-919b-be79e4f903b5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.243713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1375/2787] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:06 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:06.247878 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004494df-3ed8-4e0f-abff-31cefa5a10e7 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.258941 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004494df-3ed8-4e0f-abff-31cefa5a10e7 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.263615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09796b-865a-452d-b28b-3ebfb53c84d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.264247 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09796b-865a-452d-b28b-3ebfb53c84d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.264559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09796b-865a-452d-b28b-3ebfb53c84d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.264844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09796b-865a-452d-b28b-3ebfb53c84d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.265584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09796b-865a-452d-b28b-3ebfb53c84d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.267218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09796b-865a-452d-b28b-3ebfb53c84d4 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.278488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1413/2788] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:06.282694 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570546e1-a60c-42b2-919b-be79e4f903b5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.293874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570546e1-a60c-42b2-919b-be79e4f903b5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.298865 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c000a5f-7722-4daf-9d9d-ef83f47281ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.299539 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c000a5f-7722-4daf-9d9d-ef83f47281ef admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.299855 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c000a5f-7722-4daf-9d9d-ef83f47281ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.300174 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c000a5f-7722-4daf-9d9d-ef83f47281ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.301012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c000a5f-7722-4daf-9d9d-ef83f47281ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.302683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c000a5f-7722-4daf-9d9d-ef83f47281ef admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.314365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1376/2789] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:06.319128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09796b-865a-452d-b28b-3ebfb53c84d4 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.330410 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09796b-865a-452d-b28b-3ebfb53c84d4 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.338610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a90a-493a-465b-8d0b-20cba4c32082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.339379 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a90a-493a-465b-8d0b-20cba4c32082 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.339699 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a90a-493a-465b-8d0b-20cba4c32082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.340038 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a90a-493a-465b-8d0b-20cba4c32082 admin admin] PATH_INFO: `/v3/projects/ff4db13668114fcba874e7c27c1912df/users/fa36ebe779b7424ea977ac13092c5f9a/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.357108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1a90a-493a-465b-8d0b-20cba4c32082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4db13668114fcba874e7c27c1912df, user_id=fa36ebe779b7424ea977ac13092c5f9a, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.359840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1a90a-493a-465b-8d0b-20cba4c32082 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.466579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1414/2790] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:06 2022] PUT /identity/v3/projects/ff4db13668114fcba874e7c27c1912df/users/fa36ebe779b7424ea977ac13092c5f9a/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:06.480637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a88e68-3845-4025-bdf8-caadbab97538 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.481100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a88e68-3845-4025-bdf8-caadbab97538 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.481739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a88e68-3845-4025-bdf8-caadbab97538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.521786 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-15a88e68-3845-4025-bdf8-caadbab97538 None None] MFA Rules not processed for user `fa36ebe779b7424ea977ac13092c5f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:06.523027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-15a88e68-3845-4025-bdf8-caadbab97538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:06.594550 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1377/2791] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:06.599204 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a90a-493a-465b-8d0b-20cba4c32082 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.658296 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:06.671053 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a90a-493a-465b-8d0b-20cba4c32082 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.675826 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cc17b-e8b4-4723-ad83-46172b210f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.676442 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc17b-e8b4-4723-ad83-46172b210f56 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.676724 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc17b-e8b4-4723-ad83-46172b210f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.677103 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc17b-e8b4-4723-ad83-46172b210f56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.677890 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cc17b-e8b4-4723-ad83-46172b210f56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.680323 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cc17b-e8b4-4723-ad83-46172b210f56 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.773300 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1415/2792] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:06 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:06.778109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a88e68-3845-4025-bdf8-caadbab97538 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.790543 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a88e68-3845-4025-bdf8-caadbab97538 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.824138 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:06.857526 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae6f6b-3fb9-4b81-95c6-44387bcbacf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.858214 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae6f6b-3fb9-4b81-95c6-44387bcbacf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.858502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae6f6b-3fb9-4b81-95c6-44387bcbacf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.858784 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae6f6b-3fb9-4b81-95c6-44387bcbacf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.859459 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae6f6b-3fb9-4b81-95c6-44387bcbacf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.861460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae6f6b-3fb9-4b81-95c6-44387bcbacf5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.941371 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1378/2793] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:06 2022] POST /identity/v3/users => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:06.945603 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cc17b-e8b4-4723-ad83-46172b210f56 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.958335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cc17b-e8b4-4723-ad83-46172b210f56 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.963865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2f96d-4e6e-43ae-bf88-d63046b9cb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.966171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2f96d-4e6e-43ae-bf88-d63046b9cb9f admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.966171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2f96d-4e6e-43ae-bf88-d63046b9cb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.966171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2f96d-4e6e-43ae-bf88-d63046b9cb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.966171 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2f96d-4e6e-43ae-bf88-d63046b9cb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.967375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2f96d-4e6e-43ae-bf88-d63046b9cb9f admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.978849 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae6f6b-3fb9-4b81-95c6-44387bcbacf5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.979793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1416/2794] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:06.980884 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-01ae6f6b-3fb9-4b81-95c6-44387bcbacf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:06.983934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2f96d-4e6e-43ae-bf88-d63046b9cb9f admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.995267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae6f6b-3fb9-4b81-95c6-44387bcbacf5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.995505 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2f96d-4e6e-43ae-bf88-d63046b9cb9f admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:07.000660 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.001329 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.001633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.001918 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] PATH_INFO: `/v3/users/ec5b19159e8047fcb646b49416f926fd` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.002039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a0b5-ca5a-4b26-b6aa-50568e677e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.002598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a0b5-ca5a-4b26-b6aa-50568e677e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.002885 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a0b5-ca5a-4b26-b6aa-50568e677e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.003122 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5b19159e8047fcb646b49416f926fd)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:07.003234 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a0b5-ca5a-4b26-b6aa-50568e677e91 admin admin] PATH_INFO: `/v3/projects/70810ed234ff483db30a84395b74218a/users/6078b88e85de44378be79fc305d54ac0/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.005083 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:07.019139 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22a0b5-ca5a-4b26-b6aa-50568e677e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70810ed234ff483db30a84395b74218a, user_id=6078b88e85de44378be79fc305d54ac0, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:07.023634 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22a0b5-ca5a-4b26-b6aa-50568e677e91 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:07.094976 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1417/2795] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:06 2022] PUT /identity/v3/projects/70810ed234ff483db30a84395b74218a/users/6078b88e85de44378be79fc305d54ac0/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:07.098968 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a0b5-ca5a-4b26-b6aa-50568e677e91 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:07.164896 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a0b5-ca5a-4b26-b6aa-50568e677e91 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:07.169784 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcb0df-337d-40d8-874a-678f937ad60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.170406 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcb0df-337d-40d8-874a-678f937ad60e admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.170696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcb0df-337d-40d8-874a-678f937ad60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.170979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcb0df-337d-40d8-874a-678f937ad60e admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.171686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcb0df-337d-40d8-874a-678f937ad60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:07.173430 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcb0df-337d-40d8-874a-678f937ad60e admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:07.184338 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1418/2796] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:37:07.188385 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcb0df-337d-40d8-874a-678f937ad60e admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:07.199927 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcb0df-337d-40d8-874a-678f937ad60e admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:07.204788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686ab8b-57d5-4250-a989-9c7e5c460766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.205519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686ab8b-57d5-4250-a989-9c7e5c460766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.205811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686ab8b-57d5-4250-a989-9c7e5c460766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.206304 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686ab8b-57d5-4250-a989-9c7e5c460766 admin admin] PATH_INFO: `/v3/projects/70810ed234ff483db30a84395b74218a/users/6078b88e85de44378be79fc305d54ac0/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.208700 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686ab8b-57d5-4250-a989-9c7e5c460766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70810ed234ff483db30a84395b74218a, user_id=6078b88e85de44378be79fc305d54ac0, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:07.211517 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686ab8b-57d5-4250-a989-9c7e5c460766 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:07.222861 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1419/2797] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:07 2022] PUT /identity/v3/projects/70810ed234ff483db30a84395b74218a/users/6078b88e85de44378be79fc305d54ac0/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:07.229659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5b19159e8047fcb646b49416f926fd'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:07.233664 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1559fb-65ac-4dca-86d1-164bf0f57f81 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.236277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1559fb-65ac-4dca-86d1-164bf0f57f81 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.236277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1559fb-65ac-4dca-86d1-164bf0f57f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.243578 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5b19159e8047fcb646b49416f926fd'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:07.251198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5b19159e8047fcb646b49416f926fd'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:07.267741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-8a1559fb-65ac-4dca-86d1-164bf0f57f81 None None] MFA Rules not processed for user `6078b88e85de44378be79fc305d54ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:07.269057 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8a1559fb-65ac-4dca-86d1-164bf0f57f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:07.336488 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5b19159e8047fcb646b49416f926fd'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:07.337124 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5b19159e8047fcb646b49416f926fd'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:07.341033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1420/2798] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:07.350959 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1379/2799] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:06 2022] DELETE /identity/v3/users/ec5b19159e8047fcb646b49416f926fd => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:07.354350 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:07.354529 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a331fe-fce8-40d3-b3a2-4ca25855476b None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.354636 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a331fe-fce8-40d3-b3a2-4ca25855476b None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.354943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a331fe-fce8-40d3-b3a2-4ca25855476b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.355809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:07.380925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-70a331fe-fce8-40d3-b3a2-4ca25855476b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:07.382306 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-70a331fe-fce8-40d3-b3a2-4ca25855476b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:07.388990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:07.426576 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fdc9b-6abd-454f-a746-7b080ba1910f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:07.431129 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0dc97b-7ae7-4670-9470-4d0c2a7b3943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.432051 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0dc97b-7ae7-4670-9470-4d0c2a7b3943 service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.433341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0dc97b-7ae7-4670-9470-4d0c2a7b3943 service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.433341 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0dc97b-7ae7-4670-9470-4d0c2a7b3943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.435593 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7f0dc97b-7ae7-4670-9470-4d0c2a7b3943 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:07.461879 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1421/2800] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:07.472881 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a331fe-fce8-40d3-b3a2-4ca25855476b None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:07.479134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-70a331fe-fce8-40d3-b3a2-4ca25855476b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:07.492695 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:07.506980 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0dc97b-7ae7-4670-9470-4d0c2a7b3943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:07.509090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0dc97b-7ae7-4670-9470-4d0c2a7b3943 service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:07.518512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1380/2801] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:07.545504 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a331fe-fce8-40d3-b3a2-4ca25855476b None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:07.549177 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11514b1a-0d04-4a30-a513-d6a1a0fdd194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.549844 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11514b1a-0d04-4a30-a513-d6a1a0fdd194 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.550129 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11514b1a-0d04-4a30-a513-d6a1a0fdd194 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.550407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11514b1a-0d04-4a30-a513-d6a1a0fdd194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.560604 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11514b1a-0d04-4a30-a513-d6a1a0fdd194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:07.563148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11514b1a-0d04-4a30-a513-d6a1a0fdd194 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:07.573735 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1422/2802] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:08.034958 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0dc97b-7ae7-4670-9470-4d0c2a7b3943 service cinder] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:08.036673 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-7f0dc97b-7ae7-4670-9470-4d0c2a7b3943 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:08.054000 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0dc97b-7ae7-4670-9470-4d0c2a7b3943 service cinder] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:08.060627 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:08.061540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:08.061805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:08.066963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] PATH_INFO: `/v3/projects/fb96b2d8474943238c0939d42e492d14` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:08.066963 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb96b2d8474943238c0939d42e492d14)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:08.067359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:08.228211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb96b2d8474943238c0939d42e492d14', 'request_id': 'req-ceea18fd-8720-4939-9d02-6daec66f202c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:08.228211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb96b2d8474943238c0939d42e492d14', 'request_id': 'req-ceea18fd-8720-4939-9d02-6daec66f202c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:08.235007 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb96b2d8474943238c0939d42e492d14', 'request_id': 'req-ceea18fd-8720-4939-9d02-6daec66f202c'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:08.364585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] The token cache is being invalidate because project fb96b2d8474943238c0939d42e492d14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:08.364874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:08.365544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:08.367078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1381/2803] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:08 2022] DELETE /identity/v3/projects/fb96b2d8474943238c0939d42e492d14 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:08.371726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11514b1a-0d04-4a30-a513-d6a1a0fdd194 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:08.383094 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11514b1a-0d04-4a30-a513-d6a1a0fdd194 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:08.442840 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:08.443672 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:08.443967 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:08.444248 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] PATH_INFO: `/v3/users/341db551fa3e4c2ea98b16ea3ad991d5` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:08.446090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] RBAC: Authorizing `identity:delete_user(user_id=341db551fa3e4c2ea98b16ea3ad991d5)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:08.448036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:08.683155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '341db551fa3e4c2ea98b16ea3ad991d5'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:08.688350 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:08.696030 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '341db551fa3e4c2ea98b16ea3ad991d5'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:08.696781 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '341db551fa3e4c2ea98b16ea3ad991d5'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:08.703748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '341db551fa3e4c2ea98b16ea3ad991d5'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:08.774889 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '341db551fa3e4c2ea98b16ea3ad991d5'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:08.782007 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1423/2804] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:08 2022] DELETE /identity/v3/users/341db551fa3e4c2ea98b16ea3ad991d5 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:09.233902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:09.235464 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:09.251370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea18fd-8720-4939-9d02-6daec66f202c admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:09.319358 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:09.320013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.320384 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.320655 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] PATH_INFO: `/v3/projects/83bd52dbd811457c80f9218ab51a2255` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.321951 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83bd52dbd811457c80f9218ab51a2255)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:09.323874 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:09.464277 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83bd52dbd811457c80f9218ab51a2255', 'request_id': 'req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:09.469973 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83bd52dbd811457c80f9218ab51a2255', 'request_id': 'req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:09.477352 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83bd52dbd811457c80f9218ab51a2255', 'request_id': 'req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:09.554453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:09.558304 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:09.560326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:09.563569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] The token cache is being invalidate because project 83bd52dbd811457c80f9218ab51a2255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:09.564160 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:09.565444 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-cb1ddd1e-e4fa-49a9-bbc2-be75907c1bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:09.579907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1382/2805] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:09 2022] DELETE /identity/v3/projects/83bd52dbd811457c80f9218ab51a2255 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:09.579907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964a2d-fae3-450a-bfc8-c2e8f60a6a6e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.579907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964a2d-fae3-450a-bfc8-c2e8f60a6a6e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.579907 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964a2d-fae3-450a-bfc8-c2e8f60a6a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.603721 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6d964a2d-fae3-450a-bfc8-c2e8f60a6a6e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:09.605065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6d964a2d-fae3-450a-bfc8-c2e8f60a6a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:09.625029 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c59c0-1177-4114-a3fa-f775a34ef56d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:09.679815 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1383/2806] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:09.683096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b486d-919e-4b27-a4a4-344acb1b5f2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:09.683770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b486d-919e-4b27-a4a4-344acb1b5f2a service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.684061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b486d-919e-4b27-a4a4-344acb1b5f2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.684347 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b486d-919e-4b27-a4a4-344acb1b5f2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.686353 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7d8b486d-919e-4b27-a4a4-344acb1b5f2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:09.739934 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb72880-f0dc-45bb-aa7f-495a73ecd07b None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.740133 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb72880-f0dc-45bb-aa7f-495a73ecd07b None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.740420 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb72880-f0dc-45bb-aa7f-495a73ecd07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.754355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b486d-919e-4b27-a4a4-344acb1b5f2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:09.756299 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b486d-919e-4b27-a4a4-344acb1b5f2a service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:09.761244 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1bb72880-f0dc-45bb-aa7f-495a73ecd07b None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:09.762568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1bb72880-f0dc-45bb-aa7f-495a73ecd07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:09.766715 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1424/2807] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:09.770774 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1384/2808] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:09.775287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b486d-919e-4b27-a4a4-344acb1b5f2a service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:09.786743 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b486d-919e-4b27-a4a4-344acb1b5f2a service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:09.792281 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffb027-d54a-4813-bba6-8e9d7eb4e746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:09.792928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffb027-d54a-4813-bba6-8e9d7eb4e746 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.793219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffb027-d54a-4813-bba6-8e9d7eb4e746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.793530 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffb027-d54a-4813-bba6-8e9d7eb4e746 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.794239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ffb027-d54a-4813-bba6-8e9d7eb4e746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:09.794550 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ffb027-d54a-4813-bba6-8e9d7eb4e746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:09.796153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ffb027-d54a-4813-bba6-8e9d7eb4e746 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:09.810286 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1425/2809] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:09.814954 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb72880-f0dc-45bb-aa7f-495a73ecd07b None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:09.825005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb72880-f0dc-45bb-aa7f-495a73ecd07b None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:09.829894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c586e74-06d4-4734-bd96-73946bbff5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:09.830519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c586e74-06d4-4734-bd96-73946bbff5f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.830807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c586e74-06d4-4734-bd96-73946bbff5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.831090 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c586e74-06d4-4734-bd96-73946bbff5f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.832038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c586e74-06d4-4734-bd96-73946bbff5f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:09.834077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c586e74-06d4-4734-bd96-73946bbff5f5 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:09.950113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1385/2810] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:09 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:09.954549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffb027-d54a-4813-bba6-8e9d7eb4e746 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:09.966054 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffb027-d54a-4813-bba6-8e9d7eb4e746 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:09.996180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:10.028075 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa7113-2ba0-4c5d-bc2f-ed34d984a90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:10.028705 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa7113-2ba0-4c5d-bc2f-ed34d984a90d admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:10.028994 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa7113-2ba0-4c5d-bc2f-ed34d984a90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:10.029277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa7113-2ba0-4c5d-bc2f-ed34d984a90d admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:10.030021 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa7113-2ba0-4c5d-bc2f-ed34d984a90d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:10.031962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa7113-2ba0-4c5d-bc2f-ed34d984a90d admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:10.106886 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1426/2811] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:09 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:10.108197 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c586e74-06d4-4734-bd96-73946bbff5f5 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:10.118851 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c586e74-06d4-4734-bd96-73946bbff5f5 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:10.123540 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735a214-846b-4adc-aed5-f1e837f9dd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:10.124144 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735a214-846b-4adc-aed5-f1e837f9dd09 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:10.124428 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735a214-846b-4adc-aed5-f1e837f9dd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:10.124702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735a214-846b-4adc-aed5-f1e837f9dd09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:10.125417 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735a214-846b-4adc-aed5-f1e837f9dd09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:10.127013 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735a214-846b-4adc-aed5-f1e837f9dd09 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:10.137225 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1386/2812] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:10.141357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa7113-2ba0-4c5d-bc2f-ed34d984a90d admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:10.152090 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa7113-2ba0-4c5d-bc2f-ed34d984a90d admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:10.157737 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f1df6-c4f4-4663-8a18-2e28f671715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:10.158596 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f1df6-c4f4-4663-8a18-2e28f671715a admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:10.158897 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f1df6-c4f4-4663-8a18-2e28f671715a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:10.159186 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f1df6-c4f4-4663-8a18-2e28f671715a admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:10.160078 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f1df6-c4f4-4663-8a18-2e28f671715a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:10.162131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f1df6-c4f4-4663-8a18-2e28f671715a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:10.174272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1427/2813] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:10.178533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735a214-846b-4adc-aed5-f1e837f9dd09 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:10.189587 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735a214-846b-4adc-aed5-f1e837f9dd09 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:10.194889 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e1c88-f969-4749-abf0-a33151b1c8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:10.196219 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e1c88-f969-4749-abf0-a33151b1c8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:10.196652 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e1c88-f969-4749-abf0-a33151b1c8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:10.196940 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e1c88-f969-4749-abf0-a33151b1c8a2 admin admin] PATH_INFO: `/v3/projects/3c69d96c094b44dc8a7a3046c2f85b9a/users/71615e50848642a4833278c8b85bd61f/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:10.213257 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e1c88-f969-4749-abf0-a33151b1c8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c69d96c094b44dc8a7a3046c2f85b9a, user_id=71615e50848642a4833278c8b85bd61f, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:10.216062 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e1c88-f969-4749-abf0-a33151b1c8a2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:10.297106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1387/2814] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:10 2022] PUT /identity/v3/projects/3c69d96c094b44dc8a7a3046c2f85b9a/users/71615e50848642a4833278c8b85bd61f/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:10.309516 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158c2d2-8f56-497e-a68b-e0b92586abf1 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:10.309841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158c2d2-8f56-497e-a68b-e0b92586abf1 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:10.310138 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158c2d2-8f56-497e-a68b-e0b92586abf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:10.341148 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-f158c2d2-8f56-497e-a68b-e0b92586abf1 None None] MFA Rules not processed for user `71615e50848642a4833278c8b85bd61f`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:10.342693 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f158c2d2-8f56-497e-a68b-e0b92586abf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:10.417169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1428/2815] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:10.429353 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e1c88-f969-4749-abf0-a33151b1c8a2 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:10.431337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-613e1c88-f969-4749-abf0-a33151b1c8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:10.492755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e1c88-f969-4749-abf0-a33151b1c8a2 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:10.496504 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552f707-2e49-4dec-a45d-7bd294cac93d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4fe04b11ec412e8f4eb67c6e54b8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:10.501809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552f707-2e49-4dec-a45d-7bd294cac93d service glance] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:10.501809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552f707-2e49-4dec-a45d-7bd294cac93d service glance] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:10.501809 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552f707-2e49-4dec-a45d-7bd294cac93d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:10.506151 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552f707-2e49-4dec-a45d-7bd294cac93d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:10.508159 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552f707-2e49-4dec-a45d-7bd294cac93d service glance] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:10.517639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1388/2816] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:11.502184 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158c2d2-8f56-497e-a68b-e0b92586abf1 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:11.504329 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-f158c2d2-8f56-497e-a68b-e0b92586abf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:11.565902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158c2d2-8f56-497e-a68b-e0b92586abf1 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:11.571643 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aac2e7-f826-405c-917d-155c6fa20d46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:11.572295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac2e7-f826-405c-917d-155c6fa20d46 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:11.572583 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac2e7-f826-405c-917d-155c6fa20d46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:11.572872 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac2e7-f826-405c-917d-155c6fa20d46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:11.574919 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-53aac2e7-f826-405c-917d-155c6fa20d46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:11.629630 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:11.645297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aac2e7-f826-405c-917d-155c6fa20d46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:11.646910 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aac2e7-f826-405c-917d-155c6fa20d46 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:11.657876 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1429/2817] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:16.416264 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552f707-2e49-4dec-a45d-7bd294cac93d service glance] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:16.418617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6552f707-2e49-4dec-a45d-7bd294cac93d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:16.461357 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:16.484533 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552f707-2e49-4dec-a45d-7bd294cac93d service glance] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:16.489249 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:16.489926 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:16.497221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:16.497221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] PATH_INFO: `/v3/users/1f3729c8935041aabcd9b123ee46156b` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:16.497221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3729c8935041aabcd9b123ee46156b)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:16.497221 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:16.680674 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3729c8935041aabcd9b123ee46156b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:16.695054 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3729c8935041aabcd9b123ee46156b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:16.710899 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3729c8935041aabcd9b123ee46156b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:16.773657 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3729c8935041aabcd9b123ee46156b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:16.776813 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3729c8935041aabcd9b123ee46156b'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:16.781787 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1389/2818] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:16 2022] DELETE /identity/v3/users/1f3729c8935041aabcd9b123ee46156b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:16.786333 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dadae-3fe4-4e0d-8a9a-01a5d2504141 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:16.786820 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dadae-3fe4-4e0d-8a9a-01a5d2504141 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:16.786954 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dadae-3fe4-4e0d-8a9a-01a5d2504141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:16.808339 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-758dadae-3fe4-4e0d-8a9a-01a5d2504141 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:16.809694 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-758dadae-3fe4-4e0d-8a9a-01a5d2504141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:16.874392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1430/2819] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:16.885040 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:16.891837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:16.962735 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a9858-6b7a-479f-9566-4038fc63fdb0 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:16.966760 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa72649-cb57-49ad-9efb-749512fb0655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:16.967466 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa72649-cb57-49ad-9efb-749512fb0655 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:16.967757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa72649-cb57-49ad-9efb-749512fb0655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:16.968038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa72649-cb57-49ad-9efb-749512fb0655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:16.978724 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa72649-cb57-49ad-9efb-749512fb0655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:16.980863 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa72649-cb57-49ad-9efb-749512fb0655 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:16.991516 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1390/2820] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:17.634610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758dadae-3fe4-4e0d-8a9a-01a5d2504141 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:17.641373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-758dadae-3fe4-4e0d-8a9a-01a5d2504141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:17.652456 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758dadae-3fe4-4e0d-8a9a-01a5d2504141 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:17.658300 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:17.659701 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:17.660257 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:17.660607 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] PATH_INFO: `/v3/projects/c367d92e315848ecb75564e84edf57b3` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:17.672739 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c367d92e315848ecb75564e84edf57b3)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:17.676433 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:17.852799 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c367d92e315848ecb75564e84edf57b3', 'request_id': 'req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:17.862089 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c367d92e315848ecb75564e84edf57b3', 'request_id': 'req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:17.944909 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c367d92e315848ecb75564e84edf57b3', 'request_id': 'req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:17.959841 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] The token cache is being invalidate because project c367d92e315848ecb75564e84edf57b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:17.960142 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:17.961132 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:17.962704 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1431/2821] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:17 2022] DELETE /identity/v3/projects/c367d92e315848ecb75564e84edf57b3 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:17.967570 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa72649-cb57-49ad-9efb-749512fb0655 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:17.977541 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa72649-cb57-49ad-9efb-749512fb0655 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:18.044976 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:18.045755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:18.046262 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:18.046546 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] PATH_INFO: `/v3/users/fa36ebe779b7424ea977ac13092c5f9a` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:18.047793 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa36ebe779b7424ea977ac13092c5f9a)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:18.050001 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:18.051861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:18.235945 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa36ebe779b7424ea977ac13092c5f9a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:18.249077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa36ebe779b7424ea977ac13092c5f9a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:18.257662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa36ebe779b7424ea977ac13092c5f9a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:18.353748 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa36ebe779b7424ea977ac13092c5f9a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:18.355250 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa36ebe779b7424ea977ac13092c5f9a'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:18.367136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1391/2822] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:17 2022] DELETE /identity/v3/users/fa36ebe779b7424ea977ac13092c5f9a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:19.170487 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:19.172925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:19.210498 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5177e7-0e1b-49b1-9c1c-c05080116ae8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:19.290262 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:19.291000 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:19.291272 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:19.292287 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] PATH_INFO: `/v3/projects/ff4db13668114fcba874e7c27c1912df` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:19.294012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4db13668114fcba874e7c27c1912df)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:19.296598 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:19.317775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:19.449654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4db13668114fcba874e7c27c1912df', 'request_id': 'req-d334a56a-7ca4-444a-9598-d86679922bca'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:19.464539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4db13668114fcba874e7c27c1912df', 'request_id': 'req-d334a56a-7ca4-444a-9598-d86679922bca'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:19.616159 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4db13668114fcba874e7c27c1912df', 'request_id': 'req-d334a56a-7ca4-444a-9598-d86679922bca'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:19.631202 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] The token cache is being invalidate because project ff4db13668114fcba874e7c27c1912df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:19.631588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:19.632343 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:19.634218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1432/2823] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:19 2022] DELETE /identity/v3/projects/ff4db13668114fcba874e7c27c1912df => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:19.638426 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:19.644712 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:19.676835 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fe2c6-9afa-4ae9-b28e-f100794c10d1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:19.783377 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:19.784168 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:19.784512 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:19.785009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] PATH_INFO: `/v3/users/6078b88e85de44378be79fc305d54ac0` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:19.786491 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6078b88e85de44378be79fc305d54ac0)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:19.789027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:20.024876 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6078b88e85de44378be79fc305d54ac0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:20.058669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6078b88e85de44378be79fc305d54ac0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:20.083081 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6078b88e85de44378be79fc305d54ac0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:20.152481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6078b88e85de44378be79fc305d54ac0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:20.154053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-ef765484-7115-46d5-9e62-8e03ec83f7f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6078b88e85de44378be79fc305d54ac0'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:20.156757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:20.166198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1392/2824] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:19 2022] DELETE /identity/v3/users/6078b88e85de44378be79fc305d54ac0 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:21.108337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:21.109865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:21.123804 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334a56a-7ca4-444a-9598-d86679922bca admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:21.190752 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:21.191402 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:21.191690 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:21.191974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] PATH_INFO: `/v3/projects/70810ed234ff483db30a84395b74218a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:21.193140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70810ed234ff483db30a84395b74218a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:21.195156 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:21.290084 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70810ed234ff483db30a84395b74218a', 'request_id': 'req-c393c628-36d9-40c3-9cfc-67ab6515d613'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:21.295819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70810ed234ff483db30a84395b74218a', 'request_id': 'req-c393c628-36d9-40c3-9cfc-67ab6515d613'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:21.353372 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70810ed234ff483db30a84395b74218a', 'request_id': 'req-c393c628-36d9-40c3-9cfc-67ab6515d613'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:21.366108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] The token cache is being invalidate because project 70810ed234ff483db30a84395b74218a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:21.366334 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:21.367549 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:21.368475 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1433/2825] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:21 2022] DELETE /identity/v3/projects/70810ed234ff483db30a84395b74218a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:29.773418 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32362af2-35f4-4e4c-9c02-b2045e5a000e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:29.773917 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32362af2-35f4-4e4c-9c02-b2045e5a000e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:29.774038 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32362af2-35f4-4e4c-9c02-b2045e5a000e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:29.804632 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-32362af2-35f4-4e4c-9c02-b2045e5a000e None None] MFA Rules not processed for user `39c6f0ee9b78413eb8c5efc7adf145b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:29.805908 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-32362af2-35f4-4e4c-9c02-b2045e5a000e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:29.863217 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1393/2826] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 28 06:37:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:30.000006 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:30.001523 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:30.049297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:30.085051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c393c628-36d9-40c3-9cfc-67ab6515d613 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:30.090155 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7020d-5675-4bd6-b597-c8a6073d522b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77571de30e934d6590ff7b58a000f5e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:30.092085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7020d-5675-4bd6-b597-c8a6073d522b service swift] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:30.092085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7020d-5675-4bd6-b597-c8a6073d522b service swift] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:30.092085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7020d-5675-4bd6-b597-c8a6073d522b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:30.100500 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7020d-5675-4bd6-b597-c8a6073d522b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:30.102168 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7020d-5675-4bd6-b597-c8a6073d522b service swift] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:30.111209 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1434/2827] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 28 06:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:37:33.098044 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32362af2-35f4-4e4c-9c02-b2045e5a000e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:33.099577 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-32362af2-35f4-4e4c-9c02-b2045e5a000e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:33.176780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32362af2-35f4-4e4c-9c02-b2045e5a000e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:33.182012 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:33.183218 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:33.183507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:33.183791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] PATH_INFO: `/v3/users/9d27158da27b43f5b6bda86ef68a76ef` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:33.185116 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d27158da27b43f5b6bda86ef68a76ef)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:33.187727 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:33.386069 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d27158da27b43f5b6bda86ef68a76ef'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:33.407854 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d27158da27b43f5b6bda86ef68a76ef'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:33.415336 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d27158da27b43f5b6bda86ef68a76ef'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:33.464440 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7020d-5675-4bd6-b597-c8a6073d522b service swift] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:33.466476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-99e7020d-5675-4bd6-b597-c8a6073d522b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:33.480565 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7020d-5675-4bd6-b597-c8a6073d522b service swift] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:33.485185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:33.485860 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:33.486150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:33.486431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] PATH_INFO: `/v3/users/71615e50848642a4833278c8b85bd61f` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:33.487599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71615e50848642a4833278c8b85bd61f)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:33.489571 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:33.510074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d27158da27b43f5b6bda86ef68a76ef'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:33.514139 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-a98e5f9a-c7f2-4329-b13e-958dc79d8f25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d27158da27b43f5b6bda86ef68a76ef'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:33.518006 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1394/2828] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:33 2022] DELETE /identity/v3/users/9d27158da27b43f5b6bda86ef68a76ef => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:33.522292 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46e63b-4c41-4341-851d-f353cdd12208 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:33.522589 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46e63b-4c41-4341-851d-f353cdd12208 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:33.522873 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46e63b-4c41-4341-851d-f353cdd12208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:33.542390 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-5f46e63b-4c41-4341-851d-f353cdd12208 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:33.543661 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5f46e63b-4c41-4341-851d-f353cdd12208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:33.552180 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:33.606109 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1395/2829] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:33.616511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46e63b-4c41-4341-851d-f353cdd12208 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:33.617942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-5f46e63b-4c41-4341-851d-f353cdd12208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:33.667276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71615e50848642a4833278c8b85bd61f'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:33.679780 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46e63b-4c41-4341-851d-f353cdd12208 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:33.681118 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71615e50848642a4833278c8b85bd61f'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:33.682075 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71615e50848642a4833278c8b85bd61f'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:33.683894 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2f637-4f14-479a-92a8-915344204380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:33.685142 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2f637-4f14-479a-92a8-915344204380 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:33.685509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2f637-4f14-479a-92a8-915344204380 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:33.685729 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2f637-4f14-479a-92a8-915344204380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:33.692581 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71615e50848642a4833278c8b85bd61f'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:33.695714 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2f637-4f14-479a-92a8-915344204380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:33.697206 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2f637-4f14-479a-92a8-915344204380 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:33.710100 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1396/2830] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:33.776467 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71615e50848642a4833278c8b85bd61f'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:33.781435 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1435/2831] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:33 2022] DELETE /identity/v3/users/71615e50848642a4833278c8b85bd61f => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:33.786141 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0074e3-d1dc-47c8-a83a-5a67a120b7d2 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:33.786653 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0074e3-d1dc-47c8-a83a-5a67a120b7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:33.790409 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0074e3-d1dc-47c8-a83a-5a67a120b7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:33.824481 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ac0074e3-d1dc-47c8-a83a-5a67a120b7d2 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:33.826470 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ac0074e3-d1dc-47c8-a83a-5a67a120b7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:33.896796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1397/2832] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:33.907548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:33.908979 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:33.973131 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88403321-9586-4aac-af4c-8471c1bf85c3 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:33.977073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372343cc-0b24-4757-ba3e-ccc594399487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:33.977607 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372343cc-0b24-4757-ba3e-ccc594399487 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:33.977892 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372343cc-0b24-4757-ba3e-ccc594399487 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:33.978175 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372343cc-0b24-4757-ba3e-ccc594399487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:33.988584 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372343cc-0b24-4757-ba3e-ccc594399487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:33.990539 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372343cc-0b24-4757-ba3e-ccc594399487 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:34.003377 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1436/2833] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:34.151135 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0074e3-d1dc-47c8-a83a-5a67a120b7d2 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:34.152683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ac0074e3-d1dc-47c8-a83a-5a67a120b7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:34.225967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0074e3-d1dc-47c8-a83a-5a67a120b7d2 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:34.229725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2c6be-1731-4688-8534-e5f859fb393c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:34.230360 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2c6be-1731-4688-8534-e5f859fb393c service cinder] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:34.230670 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2c6be-1731-4688-8534-e5f859fb393c service cinder] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:34.230949 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2c6be-1731-4688-8534-e5f859fb393c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:34.233705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c2f2c6be-1731-4688-8534-e5f859fb393c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:34.245822 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=100 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:34.291105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372343cc-0b24-4757-ba3e-ccc594399487 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:34.292617 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-372343cc-0b24-4757-ba3e-ccc594399487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:34.316925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372343cc-0b24-4757-ba3e-ccc594399487 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:34.326344 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2c6be-1731-4688-8534-e5f859fb393c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:34.327546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:34.328217 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:34.328867 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2c6be-1731-4688-8534-e5f859fb393c service cinder] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:34.331073 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:34.331688 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] PATH_INFO: `/v3/projects/3eede0dda6924c4e979c7156520cd457` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:34.333279 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eede0dda6924c4e979c7156520cd457)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:34.337569 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:34.342807 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1398/2834] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:34.502716 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:34.542908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eede0dda6924c4e979c7156520cd457', 'request_id': 'req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:34.552493 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eede0dda6924c4e979c7156520cd457', 'request_id': 'req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:34.621648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eede0dda6924c4e979c7156520cd457', 'request_id': 'req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:34.642463 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] The token cache is being invalidate because project 3eede0dda6924c4e979c7156520cd457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:34.643170 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:34.643945 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:34.645962 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1437/2835] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:34 2022] DELETE /identity/v3/projects/3eede0dda6924c4e979c7156520cd457 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:34.650986 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db4cf0-11f3-4373-82a7-5329db9c21df None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:34.651527 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db4cf0-11f3-4373-82a7-5329db9c21df None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:34.651886 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db4cf0-11f3-4373-82a7-5329db9c21df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:34.680743 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b1db4cf0-11f3-4373-82a7-5329db9c21df None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:34.682598 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b1db4cf0-11f3-4373-82a7-5329db9c21df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:34.725782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:34.727368 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:34.780772 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1399/2836] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:34.789544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4df56b-a70c-4a95-bd17-bf1226ad7c8e None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:34.789847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4df56b-a70c-4a95-bd17-bf1226ad7c8e None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:34.790136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4df56b-a70c-4a95-bd17-bf1226ad7c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:34.810630 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-2a4df56b-a70c-4a95-bd17-bf1226ad7c8e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:34.811993 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2a4df56b-a70c-4a95-bd17-bf1226ad7c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:34.816931 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1bcb4-93b6-4ffd-998b-e67611a0d261 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:34.824638 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1400/2837] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:34.825660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:34.826870 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:34.827166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:34.827453 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] PATH_INFO: `/v3/projects/3c69d96c094b44dc8a7a3046c2f85b9a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:34.827783 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4df56b-a70c-4a95-bd17-bf1226ad7c8e None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:34.828748 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c69d96c094b44dc8a7a3046c2f85b9a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:34.830736 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:34.848702 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4df56b-a70c-4a95-bd17-bf1226ad7c8e None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:34.854928 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c2998-c8c2-4b51-980f-e27504ace1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:34.855558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c2998-c8c2-4b51-980f-e27504ace1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:34.855847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c2998-c8c2-4b51-980f-e27504ace1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:34.856132 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c2998-c8c2-4b51-980f-e27504ace1f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:34.857019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c2998-c8c2-4b51-980f-e27504ace1f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80318) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:34.858049 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c2998-c8c2-4b51-980f-e27504ace1f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:34.859761 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c2998-c8c2-4b51-980f-e27504ace1f8 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:34.880709 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1401/2838] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:34.884509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c2998-c8c2-4b51-980f-e27504ace1f8 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:34.910942 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c2998-c8c2-4b51-980f-e27504ace1f8 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:34.917805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117312e-0bc1-40af-8a88-c5c010f4f1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:34.918449 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117312e-0bc1-40af-8a88-c5c010f4f1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:34.918756 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117312e-0bc1-40af-8a88-c5c010f4f1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:34.919065 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117312e-0bc1-40af-8a88-c5c010f4f1d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:34.919937 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117312e-0bc1-40af-8a88-c5c010f4f1d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:34.922713 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117312e-0bc1-40af-8a88-c5c010f4f1d7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:34.994537 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c69d96c094b44dc8a7a3046c2f85b9a', 'request_id': 'req-3ac12840-b104-4e19-80ee-ca5ff25c4858'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:35.000096 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c69d96c094b44dc8a7a3046c2f85b9a', 'request_id': 'req-3ac12840-b104-4e19-80ee-ca5ff25c4858'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:35.025571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1402/2839] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:34 2022] POST /identity/v3/projects => generated 399 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:35.030047 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117312e-0bc1-40af-8a88-c5c010f4f1d7 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:35.113243 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117312e-0bc1-40af-8a88-c5c010f4f1d7 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:35.118453 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf869399-9d40-474d-91ee-62bd4381355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:35.119086 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869399-9d40-474d-91ee-62bd4381355f admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:35.119239 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c69d96c094b44dc8a7a3046c2f85b9a', 'request_id': 'req-3ac12840-b104-4e19-80ee-ca5ff25c4858'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:35.119460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869399-9d40-474d-91ee-62bd4381355f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:35.119788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869399-9d40-474d-91ee-62bd4381355f admin admin] PATH_INFO: `/v3/users` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:35.120479 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf869399-9d40-474d-91ee-62bd4381355f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:35.123097 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf869399-9d40-474d-91ee-62bd4381355f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:35.133045 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] The token cache is being invalidate because project 3c69d96c094b44dc8a7a3046c2f85b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:35.133365 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:35.134115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:35.135602 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1438/2840] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:34 2022] DELETE /identity/v3/projects/3c69d96c094b44dc8a7a3046c2f85b9a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:35.236263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:35.242487 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1403/2841] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:35 2022] POST /identity/v3/users => generated 300 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:35.247632 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:35.249441 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:35.319659 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:35.337579 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac12840-b104-4e19-80ee-ca5ff25c4858 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:35.344412 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebedeb-6694-474a-8390-f521bdea7334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:35.345087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebedeb-6694-474a-8390-f521bdea7334 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:35.345407 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebedeb-6694-474a-8390-f521bdea7334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:35.345786 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebedeb-6694-474a-8390-f521bdea7334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:35.346849 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ebedeb-6694-474a-8390-f521bdea7334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:35.348518 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ebedeb-6694-474a-8390-f521bdea7334 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:35.374278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1439/2842] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:35.378489 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf869399-9d40-474d-91ee-62bd4381355f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:35.390964 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf869399-9d40-474d-91ee-62bd4381355f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:35.397005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dadb00-5b5e-4cfb-83cc-550cddb8ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:35.397733 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dadb00-5b5e-4cfb-83cc-550cddb8ea79 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:35.398026 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dadb00-5b5e-4cfb-83cc-550cddb8ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:35.398310 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dadb00-5b5e-4cfb-83cc-550cddb8ea79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:35.399031 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dadb00-5b5e-4cfb-83cc-550cddb8ea79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:35.400773 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dadb00-5b5e-4cfb-83cc-550cddb8ea79 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:35.417406 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1404/2843] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:35.418635 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebedeb-6694-474a-8390-f521bdea7334 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:35.430092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebedeb-6694-474a-8390-f521bdea7334 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:35.436358 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006ef66-a05d-4191-abe9-453329287508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:35.437056 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006ef66-a05d-4191-abe9-453329287508 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:35.437375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006ef66-a05d-4191-abe9-453329287508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:35.437655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006ef66-a05d-4191-abe9-453329287508 admin admin] PATH_INFO: `/v3/projects/3acb504e7d5b4129bf2893a224187be1/users/6579b09d987146f29a2e358e9d59aa19/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:35.454788 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006ef66-a05d-4191-abe9-453329287508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3acb504e7d5b4129bf2893a224187be1, user_id=6579b09d987146f29a2e358e9d59aa19, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:35.457926 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006ef66-a05d-4191-abe9-453329287508 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:35.538845 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1440/2844] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:35 2022] PUT /identity/v3/projects/3acb504e7d5b4129bf2893a224187be1/users/6579b09d987146f29a2e358e9d59aa19/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:35.541911 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2c988-53f3-4418-9249-2774b49faee4 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:35.545440 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2c988-53f3-4418-9249-2774b49faee4 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:35.545771 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2c988-53f3-4418-9249-2774b49faee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:35.591624 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-70d2c988-53f3-4418-9249-2774b49faee4 None None] MFA Rules not processed for user `6579b09d987146f29a2e358e9d59aa19`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:35.592853 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-70d2c988-53f3-4418-9249-2774b49faee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:35.677234 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1405/2845] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:35.686829 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006ef66-a05d-4191-abe9-453329287508 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:35.689123 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-7006ef66-a05d-4191-abe9-453329287508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:35.750985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006ef66-a05d-4191-abe9-453329287508 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:35.755934 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170f753-b286-48b2-8a66-cfa06d4deec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:35.756753 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170f753-b286-48b2-8a66-cfa06d4deec8 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:35.756936 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170f753-b286-48b2-8a66-cfa06d4deec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:35.757399 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170f753-b286-48b2-8a66-cfa06d4deec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:35.767085 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170f753-b286-48b2-8a66-cfa06d4deec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:35.768544 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170f753-b286-48b2-8a66-cfa06d4deec8 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:35.778326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1441/2846] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:39.184072 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2c988-53f3-4418-9249-2774b49faee4 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:39.186184 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-70d2c988-53f3-4418-9249-2774b49faee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:39.275765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2c988-53f3-4418-9249-2774b49faee4 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:39.275765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:39.275765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:39.275765 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:39.276588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] PATH_INFO: `/v3/users/677cbef82d164fa8bd0a24afc4012cf7` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:39.276588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] RBAC: Authorizing `identity:delete_user(user_id=677cbef82d164fa8bd0a24afc4012cf7)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:39.276588 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:39.567882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677cbef82d164fa8bd0a24afc4012cf7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:39.585837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677cbef82d164fa8bd0a24afc4012cf7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:39.594900 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677cbef82d164fa8bd0a24afc4012cf7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:39.667244 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677cbef82d164fa8bd0a24afc4012cf7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:39.667885 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677cbef82d164fa8bd0a24afc4012cf7'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:39.670330 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:39.676042 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1406/2847] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:39 2022] DELETE /identity/v3/users/677cbef82d164fa8bd0a24afc4012cf7 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:39.968719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170f753-b286-48b2-8a66-cfa06d4deec8 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:39.984128 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170f753-b286-48b2-8a66-cfa06d4deec8 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:40.043551 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:40.044671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:40.044981 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:40.045231 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] PATH_INFO: `/v3/projects/d45f04c2af574b889ee776b6a0fa9d40` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:40.046419 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45f04c2af574b889ee776b6a0fa9d40)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:40.048326 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:40.140387 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45f04c2af574b889ee776b6a0fa9d40', 'request_id': 'req-e84c680c-a336-4726-ba0d-773ffa9216b1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:40.145938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45f04c2af574b889ee776b6a0fa9d40', 'request_id': 'req-e84c680c-a336-4726-ba0d-773ffa9216b1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:40.234776 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45f04c2af574b889ee776b6a0fa9d40', 'request_id': 'req-e84c680c-a336-4726-ba0d-773ffa9216b1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:40.236268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:40.249033 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] The token cache is being invalidate because project d45f04c2af574b889ee776b6a0fa9d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:40.249357 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:40.250036 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:40.251589 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1442/2848] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:39 2022] DELETE /identity/v3/projects/d45f04c2af574b889ee776b6a0fa9d40 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:40.255683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:40.257113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:40.270866 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cd2d2-fd0e-40ff-a8be-1a50e8bd495b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:40.336590 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:40.337164 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:40.337486 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:40.337767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] PATH_INFO: `/v3/users/93c551395aa7400181d07af92f01d518` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:40.338961 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c551395aa7400181d07af92f01d518)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:40.341749 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:40.592366 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c551395aa7400181d07af92f01d518'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:40.603950 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c551395aa7400181d07af92f01d518'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:40.611166 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c551395aa7400181d07af92f01d518'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:40.675528 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c551395aa7400181d07af92f01d518'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:40.676185 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-407b57c4-4980-4805-993a-6b28f6e548a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c551395aa7400181d07af92f01d518'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:40.682662 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1407/2849] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:40 2022] DELETE /identity/v3/users/93c551395aa7400181d07af92f01d518 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:40.960157 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:40.966219 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:40.975009 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c680c-a336-4726-ba0d-773ffa9216b1 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:41.035069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:41.035726 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:41.036011 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:41.036297 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] PATH_INFO: `/v3/projects/21203e171c8f4614b3039f5737a866ef` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:41.037587 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21203e171c8f4614b3039f5737a866ef)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:41.039555 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:41.142591 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21203e171c8f4614b3039f5737a866ef', 'request_id': 'req-70f462be-4b82-44b6-a55f-8e7eff5a92e8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.147923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21203e171c8f4614b3039f5737a866ef', 'request_id': 'req-70f462be-4b82-44b6-a55f-8e7eff5a92e8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.215289 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21203e171c8f4614b3039f5737a866ef', 'request_id': 'req-70f462be-4b82-44b6-a55f-8e7eff5a92e8'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.227823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] The token cache is being invalidate because project 21203e171c8f4614b3039f5737a866ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:41.228108 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.228746 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.230327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1443/2850] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:40 2022] DELETE /identity/v3/projects/21203e171c8f4614b3039f5737a866ef => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:45.232438 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcd864-3841-495f-af6a-4cc3da867806 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.232987 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcd864-3841-495f-af6a-4cc3da867806 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.233105 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcd864-3841-495f-af6a-4cc3da867806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.252323 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-4afcd864-3841-495f-af6a-4cc3da867806 None None] MFA Rules not processed for user `b1868c7d0ec8443b9ccb698ee02928b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:45.253558 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-4afcd864-3841-495f-af6a-4cc3da867806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:45.310233 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1408/2851] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:45.977948 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.979375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.045928 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f462be-4b82-44b6-a55f-8e7eff5a92e8 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.050744 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.051431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.051720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.052002 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] PATH_INFO: `/v3/users/6579b09d987146f29a2e358e9d59aa19` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.053185 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6579b09d987146f29a2e358e9d59aa19)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.055237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.141923 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:46.275373 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6579b09d987146f29a2e358e9d59aa19'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:46.290958 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6579b09d987146f29a2e358e9d59aa19'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:46.291714 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6579b09d987146f29a2e358e9d59aa19'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:46.300615 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6579b09d987146f29a2e358e9d59aa19'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:46.369869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6579b09d987146f29a2e358e9d59aa19'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:46.376719 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1444/2852] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:45 2022] DELETE /identity/v3/users/6579b09d987146f29a2e358e9d59aa19 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:46.381757 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec351e16-1c12-4c67-8288-29276bf54a75 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.382266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec351e16-1c12-4c67-8288-29276bf54a75 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.382266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec351e16-1c12-4c67-8288-29276bf54a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.402232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-ec351e16-1c12-4c67-8288-29276bf54a75 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:46.403430 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ec351e16-1c12-4c67-8288-29276bf54a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.421687 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:46.463258 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1409/2853] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:46.474012 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.475422 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.535335 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa060f8-db44-4c49-8e56-1370ea904a3a admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.538977 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0931bb-67a2-476b-84fd-24519289ada8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.539597 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0931bb-67a2-476b-84fd-24519289ada8 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.539908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0931bb-67a2-476b-84fd-24519289ada8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.540183 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0931bb-67a2-476b-84fd-24519289ada8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.549619 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0931bb-67a2-476b-84fd-24519289ada8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.551060 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0931bb-67a2-476b-84fd-24519289ada8 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.561225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1445/2854] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:46.968148 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec351e16-1c12-4c67-8288-29276bf54a75 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.969704 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-ec351e16-1c12-4c67-8288-29276bf54a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.982585 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec351e16-1c12-4c67-8288-29276bf54a75 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.987367 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.988009 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.988294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.988573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] PATH_INFO: `/v3/projects/3acb504e7d5b4129bf2893a224187be1` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.989774 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3acb504e7d5b4129bf2893a224187be1)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.991785 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.110582 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3acb504e7d5b4129bf2893a224187be1', 'request_id': 'req-7304b74c-80d9-4815-afd6-311c225bb884'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:47.116319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3acb504e7d5b4129bf2893a224187be1', 'request_id': 'req-7304b74c-80d9-4815-afd6-311c225bb884'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:47.122639 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3acb504e7d5b4129bf2893a224187be1', 'request_id': 'req-7304b74c-80d9-4815-afd6-311c225bb884'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:47.203147 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] The token cache is being invalidate because project 3acb504e7d5b4129bf2893a224187be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:47.203460 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:47.204077 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-7304b74c-80d9-4815-afd6-311c225bb884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:47.205633 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1410/2855] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:46 2022] DELETE /identity/v3/projects/3acb504e7d5b4129bf2893a224187be1 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:47.210965 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d337e8-0873-46b9-8060-9af6405aeb8e None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.211437 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d337e8-0873-46b9-8060-9af6405aeb8e None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.211548 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d337e8-0873-46b9-8060-9af6405aeb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.231371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-70d337e8-0873-46b9-8060-9af6405aeb8e None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:47.233542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-70d337e8-0873-46b9-8060-9af6405aeb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:47.293092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1446/2856] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:47.303027 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f8d2-b2f8-49fd-adf2-f7b61a2d3a18 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.303319 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f8d2-b2f8-49fd-adf2-f7b61a2d3a18 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.303604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f8d2-b2f8-49fd-adf2-f7b61a2d3a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.324519 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-b944f8d2-b2f8-49fd-adf2-f7b61a2d3a18 None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:47.325806 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b944f8d2-b2f8-49fd-adf2-f7b61a2d3a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:47.336289 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1411/2857] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:47.340777 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d337e8-0873-46b9-8060-9af6405aeb8e None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.354169 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d337e8-0873-46b9-8060-9af6405aeb8e None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.359823 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61fa1c-175c-4604-81ae-b03f0b709500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.360712 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61fa1c-175c-4604-81ae-b03f0b709500 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.362192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61fa1c-175c-4604-81ae-b03f0b709500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.362192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61fa1c-175c-4604-81ae-b03f0b709500 admin admin] PATH_INFO: `/v3/domains` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.362192 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61fa1c-175c-4604-81ae-b03f0b709500 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80317) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:47.362631 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61fa1c-175c-4604-81ae-b03f0b709500 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.365865 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61fa1c-175c-4604-81ae-b03f0b709500 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.377126 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1447/2858] 10.222.0.69 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:47.381725 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f8d2-b2f8-49fd-adf2-f7b61a2d3a18 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.391944 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f8d2-b2f8-49fd-adf2-f7b61a2d3a18 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.396739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183e43a-e408-4df0-a8f1-ec6588ee41d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.397416 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183e43a-e408-4df0-a8f1-ec6588ee41d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.397734 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183e43a-e408-4df0-a8f1-ec6588ee41d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.398008 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183e43a-e408-4df0-a8f1-ec6588ee41d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.398701 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183e43a-e408-4df0-a8f1-ec6588ee41d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.400683 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183e43a-e408-4df0-a8f1-ec6588ee41d7 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.532935 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1412/2859] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:47 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:47.537158 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61fa1c-175c-4604-81ae-b03f0b709500 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.547651 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61fa1c-175c-4604-81ae-b03f0b709500 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.578808 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:47.606069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187ec8e-8ef5-48fc-ac0d-2ed21fc81f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.606686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187ec8e-8ef5-48fc-ac0d-2ed21fc81f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.607051 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187ec8e-8ef5-48fc-ac0d-2ed21fc81f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.607439 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187ec8e-8ef5-48fc-ac0d-2ed21fc81f29 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.608134 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187ec8e-8ef5-48fc-ac0d-2ed21fc81f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.610153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187ec8e-8ef5-48fc-ac0d-2ed21fc81f29 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.719682 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1448/2860] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:47 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:47.724074 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183e43a-e408-4df0-a8f1-ec6588ee41d7 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.735080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183e43a-e408-4df0-a8f1-ec6588ee41d7 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.739636 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a692f04-624a-429f-bd58-34029b59167f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.740359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a692f04-624a-429f-bd58-34029b59167f admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.740637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a692f04-624a-429f-bd58-34029b59167f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.740912 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a692f04-624a-429f-bd58-34029b59167f admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.741637 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a692f04-624a-429f-bd58-34029b59167f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.743238 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a692f04-624a-429f-bd58-34029b59167f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.753726 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1413/2861] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:47.757937 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187ec8e-8ef5-48fc-ac0d-2ed21fc81f29 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.768663 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187ec8e-8ef5-48fc-ac0d-2ed21fc81f29 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.773355 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e2196-dae1-4115-93bd-04ffaef75007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.773904 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e2196-dae1-4115-93bd-04ffaef75007 admin admin] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.774220 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e2196-dae1-4115-93bd-04ffaef75007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.774525 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e2196-dae1-4115-93bd-04ffaef75007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.775225 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e2196-dae1-4115-93bd-04ffaef75007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.776813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e2196-dae1-4115-93bd-04ffaef75007 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.788900 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1449/2862] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:47.792275 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a692f04-624a-429f-bd58-34029b59167f admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.802507 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a692f04-624a-429f-bd58-34029b59167f admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.807079 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7881d2-9ae4-4d77-9d03-3d2774fe51f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.807737 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7881d2-9ae4-4d77-9d03-3d2774fe51f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.808019 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7881d2-9ae4-4d77-9d03-3d2774fe51f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.808294 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7881d2-9ae4-4d77-9d03-3d2774fe51f1 admin admin] PATH_INFO: `/v3/projects/3ce97bbb75d4412e90231e4da14840bc/users/95a22e1003d74332ba4f0ee84b6422e4/roles/c4fc73849d664b2588bd2de7372aa839` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.826251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7881d2-9ae4-4d77-9d03-3d2774fe51f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce97bbb75d4412e90231e4da14840bc, user_id=95a22e1003d74332ba4f0ee84b6422e4, role_id=c4fc73849d664b2588bd2de7372aa839)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.829201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7881d2-9ae4-4d77-9d03-3d2774fe51f1 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.944847 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1414/2863] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:47 2022] PUT /identity/v3/projects/3ce97bbb75d4412e90231e4da14840bc/users/95a22e1003d74332ba4f0ee84b6422e4/roles/c4fc73849d664b2588bd2de7372aa839 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:47.950088 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65559a-8505-43b9-a889-a1938d2f1e9a None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.950384 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65559a-8505-43b9-a889-a1938d2f1e9a None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.951312 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65559a-8505-43b9-a889-a1938d2f1e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.978165 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-4b65559a-8505-43b9-a889-a1938d2f1e9a None None] MFA Rules not processed for user `b42a77304cdd4ceda1f777fdc9b5ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:47.979371 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-4b65559a-8505-43b9-a889-a1938d2f1e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:48.049444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1450/2864] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:48.059156 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7881d2-9ae4-4d77-9d03-3d2774fe51f1 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:48.061483 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-2a7881d2-9ae4-4d77-9d03-3d2774fe51f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:48.109196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:48.120952 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7881d2-9ae4-4d77-9d03-3d2774fe51f1 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:48.125365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541cc79-8add-4268-b96c-b305deb8287f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:48.126056 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541cc79-8add-4268-b96c-b305deb8287f service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:48.126347 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541cc79-8add-4268-b96c-b305deb8287f service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:48.126659 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541cc79-8add-4268-b96c-b305deb8287f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:48.135966 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541cc79-8add-4268-b96c-b305deb8287f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:48.137522 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541cc79-8add-4268-b96c-b305deb8287f service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:48.147777 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1415/2865] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:54.218163 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63022f3-8e1c-4c2c-9878-f7d10335b0f7 None None] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:54.219115 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63022f3-8e1c-4c2c-9878-f7d10335b0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:54.219574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63022f3-8e1c-4c2c-9878-f7d10335b0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:54.255608 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.auth.core [None req-e63022f3-8e1c-4c2c-9878-f7d10335b0f7 None None] MFA Rules not processed for user `95a22e1003d74332ba4f0ee84b6422e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80317) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:54.256882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e63022f3-8e1c-4c2c-9878-f7d10335b0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:54.348166 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1451/2866] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:37:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:54.352102 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541cc79-8add-4268-b96c-b305deb8287f service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:54.370345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541cc79-8add-4268-b96c-b305deb8287f service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:54.378118 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f1671-72ca-4c21-bccd-1046cf7029d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:54.379307 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f1671-72ca-4c21-bccd-1046cf7029d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:54.380075 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f1671-72ca-4c21-bccd-1046cf7029d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:54.381071 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f1671-72ca-4c21-bccd-1046cf7029d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:54.381973 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f1671-72ca-4c21-bccd-1046cf7029d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:54.384728 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f1671-72ca-4c21-bccd-1046cf7029d4 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:54.509050 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1416/2867] 10.222.0.69 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:54 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:54.513769 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63022f3-8e1c-4c2c-9878-f7d10335b0f7 None None] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:54.525237 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63022f3-8e1c-4c2c-9878-f7d10335b0f7 None None] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:54.569133 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:54.597966 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515868a7-0946-4ab3-b3fc-7cad28ccd070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:54.598797 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515868a7-0946-4ab3-b3fc-7cad28ccd070 admin admin] REQUEST_METHOD: `POST` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:54.599153 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515868a7-0946-4ab3-b3fc-7cad28ccd070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:54.599443 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515868a7-0946-4ab3-b3fc-7cad28ccd070 admin admin] PATH_INFO: `/v3/users` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:54.600141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515868a7-0946-4ab3-b3fc-7cad28ccd070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:54.602331 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515868a7-0946-4ab3-b3fc-7cad28ccd070 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:54.744396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1452/2868] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:54 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:54.749196 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f1671-72ca-4c21-bccd-1046cf7029d4 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:54.760615 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f1671-72ca-4c21-bccd-1046cf7029d4 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:54.765787 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00bd0-cc60-40b6-a4e0-f7b6f175ae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:54.766552 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00bd0-cc60-40b6-a4e0-f7b6f175ae44 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:54.766904 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00bd0-cc60-40b6-a4e0-f7b6f175ae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:54.767247 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00bd0-cc60-40b6-a4e0-f7b6f175ae44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:54.768198 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec00bd0-cc60-40b6-a4e0-f7b6f175ae44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:54.769955 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec00bd0-cc60-40b6-a4e0-f7b6f175ae44 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:54.784741 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1417/2869] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:54.789066 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515868a7-0946-4ab3-b3fc-7cad28ccd070 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:54.802354 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515868a7-0946-4ab3-b3fc-7cad28ccd070 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:54.808996 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82842d49-49d7-456c-a5e6-d888a0e2bdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:54.810627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82842d49-49d7-456c-a5e6-d888a0e2bdda admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:54.810627 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82842d49-49d7-456c-a5e6-d888a0e2bdda admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:54.811025 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82842d49-49d7-456c-a5e6-d888a0e2bdda admin admin] PATH_INFO: `/v3/projects/240811071c1541a4896bd8c1c14442bf/users/74467350826d43dc869c01ead878842a/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:54.832819 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82842d49-49d7-456c-a5e6-d888a0e2bdda admin admin] RBAC: Authorizing `identity:create_grant(project_id=240811071c1541a4896bd8c1c14442bf, user_id=74467350826d43dc869c01ead878842a, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:54.836461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82842d49-49d7-456c-a5e6-d888a0e2bdda admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:54.971782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1453/2870] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:54 2022] PUT /identity/v3/projects/240811071c1541a4896bd8c1c14442bf/users/74467350826d43dc869c01ead878842a/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:54.975805 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00bd0-cc60-40b6-a4e0-f7b6f175ae44 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:54.987569 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00bd0-cc60-40b6-a4e0-f7b6f175ae44 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:55.055121 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be331bf-aaed-4054-a122-2ec54b080999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:55.055796 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be331bf-aaed-4054-a122-2ec54b080999 admin admin] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:55.056080 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be331bf-aaed-4054-a122-2ec54b080999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:55.056378 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be331bf-aaed-4054-a122-2ec54b080999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:55.057112 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be331bf-aaed-4054-a122-2ec54b080999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:55.058781 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be331bf-aaed-4054-a122-2ec54b080999 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:55.072970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1418/2871] 10.222.0.69 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:55.078181 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82842d49-49d7-456c-a5e6-d888a0e2bdda admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:55.094315 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82842d49-49d7-456c-a5e6-d888a0e2bdda admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:55.099268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b43c11-8590-48c1-a31f-5fe0898059d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:55.099997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b43c11-8590-48c1-a31f-5fe0898059d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:55.100285 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b43c11-8590-48c1-a31f-5fe0898059d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:55.100543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b43c11-8590-48c1-a31f-5fe0898059d2 admin admin] PATH_INFO: `/v3/projects/240811071c1541a4896bd8c1c14442bf/users/74467350826d43dc869c01ead878842a/roles/728f3655cf7a47fc9e88157b0dd7dca4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:55.102801 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b43c11-8590-48c1-a31f-5fe0898059d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240811071c1541a4896bd8c1c14442bf, user_id=74467350826d43dc869c01ead878842a, role_id=728f3655cf7a47fc9e88157b0dd7dca4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:55.105938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b43c11-8590-48c1-a31f-5fe0898059d2 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:55.118821 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1454/2872] 10.222.0.69 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:55 2022] PUT /identity/v3/projects/240811071c1541a4896bd8c1c14442bf/users/74467350826d43dc869c01ead878842a/roles/728f3655cf7a47fc9e88157b0dd7dca4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:01.210837 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fbee6-4894-4075-ad58-8e5f5095e81a None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:01.211361 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fbee6-4894-4075-ad58-8e5f5095e81a None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:01.211477 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fbee6-4894-4075-ad58-8e5f5095e81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:01.245106 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-1f9fbee6-4894-4075-ad58-8e5f5095e81a None None] MFA Rules not processed for user `74467350826d43dc869c01ead878842a`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:01.246413 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1f9fbee6-4894-4075-ad58-8e5f5095e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:01.325664 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1419/2873] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:38:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:01.333974 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b43c11-8590-48c1-a31f-5fe0898059d2 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:01.335774 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d2b43c11-8590-48c1-a31f-5fe0898059d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:01.428039 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b43c11-8590-48c1-a31f-5fe0898059d2 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:01.436040 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17379fc0-73d9-4ed8-8fa7-bcf616bd0c21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:01.436793 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17379fc0-73d9-4ed8-8fa7-bcf616bd0c21 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:01.437092 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17379fc0-73d9-4ed8-8fa7-bcf616bd0c21 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:01.437396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17379fc0-73d9-4ed8-8fa7-bcf616bd0c21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:01.509476 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17379fc0-73d9-4ed8-8fa7-bcf616bd0c21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:01.511102 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17379fc0-73d9-4ed8-8fa7-bcf616bd0c21 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:01.523824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1455/2874] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:04.528982 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97f73e-f152-40f5-8c4c-d1f2125faac1 None None] REQUEST_METHOD: `POST` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:04.529812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97f73e-f152-40f5-8c4c-d1f2125faac1 None None] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:04.529812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97f73e-f152-40f5-8c4c-d1f2125faac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:04.551991 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.auth.core [None req-6a97f73e-f152-40f5-8c4c-d1f2125faac1 None None] MFA Rules not processed for user `95a22e1003d74332ba4f0ee84b6422e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80318) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:04.553359 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6a97f73e-f152-40f5-8c4c-d1f2125faac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:04.561617 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1420/2875] 10.222.0.69 () {58 vars in 958 bytes} [Thu Jul 28 06:38:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:08.676490 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17379fc0-73d9-4ed8-8fa7-bcf616bd0c21 service nova] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:08.713528 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:08.736925 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17379fc0-73d9-4ed8-8fa7-bcf616bd0c21 service nova] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:08.740652 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f330821-2a43-411d-9438-b5d8bb944d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:08.741284 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f330821-2a43-411d-9438-b5d8bb944d93 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.741610 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f330821-2a43-411d-9438-b5d8bb944d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.741893 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f330821-2a43-411d-9438-b5d8bb944d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.743785 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9f330821-2a43-411d-9438-b5d8bb944d93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:08.759059 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f330821-2a43-411d-9438-b5d8bb944d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:08.760560 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f330821-2a43-411d-9438-b5d8bb944d93 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.772557 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1456/2876] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:09.091005 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97f73e-f152-40f5-8c4c-d1f2125faac1 None None] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.092599 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6a97f73e-f152-40f5-8c4c-d1f2125faac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:09.104970 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97f73e-f152-40f5-8c4c-d1f2125faac1 None None] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.108622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ea2cf-ef99-4b04-b09b-de8b98792220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.109232 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ea2cf-ef99-4b04-b09b-de8b98792220 service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.109549 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ea2cf-ef99-4b04-b09b-de8b98792220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.109829 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ea2cf-ef99-4b04-b09b-de8b98792220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.111571 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-802ea2cf-ef99-4b04-b09b-de8b98792220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:09.138878 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:09.179795 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ea2cf-ef99-4b04-b09b-de8b98792220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.181345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ea2cf-ef99-4b04-b09b-de8b98792220 service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.192058 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1421/2877] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:18.782908 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=12 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:19.323978 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=18 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:20.235564 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f330821-2a43-411d-9438-b5d8bb944d93 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.236450 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-9f330821-2a43-411d-9438-b5d8bb944d93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:20.297864 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f330821-2a43-411d-9438-b5d8bb944d93 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.301741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbf07b-d603-4ba2-a32b-f425af874724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbbbf34c238d44db8588d56f35827390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.302375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbf07b-d603-4ba2-a32b-f425af874724 service cinder] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.302660 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbf07b-d603-4ba2-a32b-f425af874724 service cinder] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.302940 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbf07b-d603-4ba2-a32b-f425af874724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.304728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-e5cbf07b-d603-4ba2-a32b-f425af874724 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:20.319396 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbf07b-d603-4ba2-a32b-f425af874724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.320853 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbf07b-d603-4ba2-a32b-f425af874724 service cinder] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.335136 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1457/2878] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:30.327431 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=26 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:43.588407 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ea2cf-ef99-4b04-b09b-de8b98792220 service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:43.597946 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ea2cf-ef99-4b04-b09b-de8b98792220 service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.601581 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8fc59-e9d3-462b-9c31-17382ec3f4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.602189 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8fc59-e9d3-462b-9c31-17382ec3f4cb service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.602476 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8fc59-e9d3-462b-9c31-17382ec3f4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.602755 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8fc59-e9d3-462b-9c31-17382ec3f4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.611722 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8fc59-e9d3-462b-9c31-17382ec3f4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.613365 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8fc59-e9d3-462b-9c31-17382ec3f4cb service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.622893 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1422/2879] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:45.843111 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbf07b-d603-4ba2-a32b-f425af874724 service cinder] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:45.852741 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbf07b-d603-4ba2-a32b-f425af874724 service cinder] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:45.856366 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f0350-a534-4241-8126-9304344a7d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:45.856990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0350-a534-4241-8126-9304344a7d71 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:45.857276 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0350-a534-4241-8126-9304344a7d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:45.857588 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0350-a534-4241-8126-9304344a7d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:45.859392 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-538f0350-a534-4241-8126-9304344a7d71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:45.870086 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f0350-a534-4241-8126-9304344a7d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:45.871538 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f0350-a534-4241-8126-9304344a7d71 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:45.880461 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1458/2880] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:53.618705 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:55.875983 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=6 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:39.408337 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8fc59-e9d3-462b-9c31-17382ec3f4cb service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:39.409948 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-34b8fc59-e9d3-462b-9c31-17382ec3f4cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:39.426708 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8fc59-e9d3-462b-9c31-17382ec3f4cb service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:39.432358 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737fc48-2826-429a-b988-fa1700630499 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:39.432971 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737fc48-2826-429a-b988-fa1700630499 service nova] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:39.433251 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737fc48-2826-429a-b988-fa1700630499 service nova] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:39.433565 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737fc48-2826-429a-b988-fa1700630499 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:39.435902 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6737fc48-2826-429a-b988-fa1700630499 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:39.453094 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737fc48-2826-429a-b988-fa1700630499 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:39.454806 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737fc48-2826-429a-b988-fa1700630499 service nova] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:39.470229 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1423/2881] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:49.464745 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:52.775444 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f0350-a534-4241-8126-9304344a7d71 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:52.784686 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f0350-a534-4241-8126-9304344a7d71 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:52.788295 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87851ec8-e584-4ab3-978c-75a522f324f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:52.788901 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87851ec8-e584-4ab3-978c-75a522f324f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:52.789180 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87851ec8-e584-4ab3-978c-75a522f324f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:52.789519 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87851ec8-e584-4ab3-978c-75a522f324f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:52.791304 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-87851ec8-e584-4ab3-978c-75a522f324f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:52.856508 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87851ec8-e584-4ab3-978c-75a522f324f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:52.858811 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87851ec8-e584-4ab3-978c-75a522f324f4 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:52.869013 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1459/2882] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:00.067739 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737fc48-2826-429a-b988-fa1700630499 service nova] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:00.069266 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-6737fc48-2826-429a-b988-fa1700630499 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:00.085200 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737fc48-2826-429a-b988-fa1700630499 service nova] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:00.089983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:00.090770 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:00.091053 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:00.091335 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] PATH_INFO: `/v3/users/b1868c7d0ec8443b9ccb698ee02928b6` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:00.093502 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1868c7d0ec8443b9ccb698ee02928b6)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:00.095446 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:00.285551 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1868c7d0ec8443b9ccb698ee02928b6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:00.298215 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1868c7d0ec8443b9ccb698ee02928b6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:00.305138 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1868c7d0ec8443b9ccb698ee02928b6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:00.379046 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1868c7d0ec8443b9ccb698ee02928b6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:00.379672 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1868c7d0ec8443b9ccb698ee02928b6'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:00.387187 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1424/2883] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:40:00 2022] DELETE /identity/v3/users/b1868c7d0ec8443b9ccb698ee02928b6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:40:00.627105 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87851ec8-e584-4ab3-978c-75a522f324f4 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:00.640750 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87851ec8-e584-4ab3-978c-75a522f324f4 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:00.699006 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:00.699654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:00.699943 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:00.700268 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] PATH_INFO: `/v3/projects/47b406d9a12c461a84aa0331d2890990` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:00.702342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=47b406d9a12c461a84aa0331d2890990)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:00.704599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:00.804342 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47b406d9a12c461a84aa0331d2890990', 'request_id': 'req-0c18a03b-1958-4998-b58b-1ab33dc30b9b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:00.810303 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47b406d9a12c461a84aa0331d2890990', 'request_id': 'req-0c18a03b-1958-4998-b58b-1ab33dc30b9b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:00.882236 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47b406d9a12c461a84aa0331d2890990', 'request_id': 'req-0c18a03b-1958-4998-b58b-1ab33dc30b9b'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:00.894524 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] The token cache is being invalidate because project 47b406d9a12c461a84aa0331d2890990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:40:00.894817 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:00.895459 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:00.896990 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1460/2884] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:40:00 2022] DELETE /identity/v3/projects/47b406d9a12c461a84aa0331d2890990 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:40:00.901125 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:00.902861 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:00.916681 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22288967-606d-4eb8-b436-111adef3e78b admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:00.973573 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:00.974255 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:00.974547 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:00.974833 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] PATH_INFO: `/v3/users/b28a7273be4947a18f690277ca61575e` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:00.976020 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b28a7273be4947a18f690277ca61575e)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:00.978004 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:01.158514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b28a7273be4947a18f690277ca61575e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:01.171167 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b28a7273be4947a18f690277ca61575e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:01.177994 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b28a7273be4947a18f690277ca61575e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:01.179791 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:01.247543 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b28a7273be4947a18f690277ca61575e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:01.248263 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b28a7273be4947a18f690277ca61575e'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:01.254564 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1425/2885] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:40:00 2022] DELETE /identity/v3/users/b28a7273be4947a18f690277ca61575e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:40:01.491076 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:01.492720 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:01.506648 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18a03b-1958-4998-b58b-1ab33dc30b9b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:01.528317 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:01.567218 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:01.567873 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:01.568164 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:01.568513 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] PATH_INFO: `/v3/projects/940cf692c0554607a4d43de7c23a22a2` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:01.569654 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=940cf692c0554607a4d43de7c23a22a2)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:01.571601 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:01.700488 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940cf692c0554607a4d43de7c23a22a2', 'request_id': 'req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:01.706574 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940cf692c0554607a4d43de7c23a22a2', 'request_id': 'req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:01.773375 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940cf692c0554607a4d43de7c23a22a2', 'request_id': 'req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:01.785082 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] The token cache is being invalidate because project 940cf692c0554607a4d43de7c23a22a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80317) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:40:01.785411 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:01.786043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:01.787543 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1461/2886] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:40:01 2022] DELETE /identity/v3/projects/940cf692c0554607a4d43de7c23a22a2 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:40:11.304270 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:11.782069 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:56.234509 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:56.236612 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:56.297439 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779561d6-704f-4ef1-8bcf-9bab5bf2c44e admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:56.346939 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c093db-ff26-4a4e-840d-faeb015904dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:56.347572 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c093db-ff26-4a4e-840d-faeb015904dd service neutron] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.347858 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c093db-ff26-4a4e-840d-faeb015904dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.348136 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c093db-ff26-4a4e-840d-faeb015904dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.349976 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c7c093db-ff26-4a4e-840d-faeb015904dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:56.360511 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c093db-ff26-4a4e-840d-faeb015904dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:56.362028 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c093db-ff26-4a4e-840d-faeb015904dd service neutron] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:56.370602 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1426/2887] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:56.449818 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:56.451824 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:56.464327 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2eb4b-f3ea-4915-9b2f-9286dc359eb1 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:56.468235 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b894c3-9466-4390-9f1a-7d7e34292cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05574eace38f4b6d945c7b1927fdadd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:56.468869 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b894c3-9466-4390-9f1a-7d7e34292cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.469190 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b894c3-9466-4390-9f1a-7d7e34292cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.469511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b894c3-9466-4390-9f1a-7d7e34292cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.471283 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-76b894c3-9466-4390-9f1a-7d7e34292cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:56.542542 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b894c3-9466-4390-9f1a-7d7e34292cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:56.543997 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b894c3-9466-4390-9f1a-7d7e34292cf0 service neutron] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:56.555599 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1462/2888] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:56.623290 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c093db-ff26-4a4e-840d-faeb015904dd service neutron] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:56.625211 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c7c093db-ff26-4a4e-840d-faeb015904dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:56.694983 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c093db-ff26-4a4e-840d-faeb015904dd service neutron] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:56.699888 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a946a2-ed90-483d-b5a3-a0eff873f719 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:56.700768 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a946a2-ed90-483d-b5a3-a0eff873f719 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.701048 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a946a2-ed90-483d-b5a3-a0eff873f719 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.701345 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a946a2-ed90-483d-b5a3-a0eff873f719 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.703689 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-c8a946a2-ed90-483d-b5a3-a0eff873f719 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:56.715241 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a946a2-ed90-483d-b5a3-a0eff873f719 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:56.716830 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a946a2-ed90-483d-b5a3-a0eff873f719 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:56.727149 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1427/2889] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:56.974063 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b894c3-9466-4390-9f1a-7d7e34292cf0 service neutron] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:56.977066 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-76b894c3-9466-4390-9f1a-7d7e34292cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:56.989174 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b894c3-9466-4390-9f1a-7d7e34292cf0 service neutron] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:56.994738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24930d-c819-45b6-9ece-613b1716c062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:56.995381 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24930d-c819-45b6-9ece-613b1716c062 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.995662 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24930d-c819-45b6-9ece-613b1716c062 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.995938 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24930d-c819-45b6-9ece-613b1716c062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.997697 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-0e24930d-c819-45b6-9ece-613b1716c062 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:57.009813 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24930d-c819-45b6-9ece-613b1716c062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:57.011454 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24930d-c819-45b6-9ece-613b1716c062 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:57.023091 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1463/2890] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:41:06.722823 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=78 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:41:07.018882 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=40 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:41:18.228943 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a946a2-ed90-483d-b5a3-a0eff873f719 service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:41:18.247035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a946a2-ed90-483d-b5a3-a0eff873f719 service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:41:18.247035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed37ae-ca8b-4eb3-9fe0-011f5e20ab15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:41:18.247035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed37ae-ca8b-4eb3-9fe0-011f5e20ab15 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:41:18.247035 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed37ae-ca8b-4eb3-9fe0-011f5e20ab15 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:41:18.247514 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed37ae-ca8b-4eb3-9fe0-011f5e20ab15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:41:18.248967 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b0ed37ae-ca8b-4eb3-9fe0-011f5e20ab15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:41:18.261692 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed37ae-ca8b-4eb3-9fe0-011f5e20ab15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:41:18.263395 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed37ae-ca8b-4eb3-9fe0-011f5e20ab15 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:41:18.275393 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1428/2891] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:41:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:41:28.270463 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=6 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:22.237337 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24930d-c819-45b6-9ece-613b1716c062 service nova] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:22.250141 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24930d-c819-45b6-9ece-613b1716c062 service nova] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:22.256696 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070354e-fd34-44bd-84ea-a4b1293aa852 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5706aa2d6bf54f05badf9588f51f4677', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:22.257770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070354e-fd34-44bd-84ea-a4b1293aa852 service nova] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:22.257770 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070354e-fd34-44bd-84ea-a4b1293aa852 service nova] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:22.258019 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070354e-fd34-44bd-84ea-a4b1293aa852 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:22.259728 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8070354e-fd34-44bd-84ea-a4b1293aa852 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:22.273264 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070354e-fd34-44bd-84ea-a4b1293aa852 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:22.277477 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070354e-fd34-44bd-84ea-a4b1293aa852 service nova] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:22.287623 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1464/2892] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:29.963669 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed37ae-ca8b-4eb3-9fe0-011f5e20ab15 service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:29.965296 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-b0ed37ae-ca8b-4eb3-9fe0-011f5e20ab15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:29.979326 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed37ae-ca8b-4eb3-9fe0-011f5e20ab15 service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:29.984651 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a4bca-f543-4c0b-b277-493dc11061d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:29.985427 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a4bca-f543-4c0b-b277-493dc11061d5 service placement] REQUEST_METHOD: `GET` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:29.985788 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a4bca-f543-4c0b-b277-493dc11061d5 service placement] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:29.986139 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a4bca-f543-4c0b-b277-493dc11061d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:29.987975 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-3d3a4bca-f543-4c0b-b277-493dc11061d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:30.000113 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a4bca-f543-4c0b-b277-493dc11061d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:30.001758 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a4bca-f543-4c0b-b277-493dc11061d5 service placement] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:30.013370 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1429/2893] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:32.281278 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=6 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:33.863043 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070354e-fd34-44bd-84ea-a4b1293aa852 service nova] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:33.864854 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-8070354e-fd34-44bd-84ea-a4b1293aa852 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:33.933695 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070354e-fd34-44bd-84ea-a4b1293aa852 service nova] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:33.938332 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:33.938985 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:33.939277 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:33.939563 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] PATH_INFO: `/v3/users/95a22e1003d74332ba4f0ee84b6422e4` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:33.940759 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95a22e1003d74332ba4f0ee84b6422e4)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:33.943006 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:34.172920 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95a22e1003d74332ba4f0ee84b6422e4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:34.184902 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95a22e1003d74332ba4f0ee84b6422e4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:34.185559 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95a22e1003d74332ba4f0ee84b6422e4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:34.192546 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95a22e1003d74332ba4f0ee84b6422e4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:34.284061 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95a22e1003d74332ba4f0ee84b6422e4'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:34.290780 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1465/2894] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:42:33 2022] DELETE /identity/v3/users/95a22e1003d74332ba4f0ee84b6422e4 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:42:34.551551 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a4bca-f543-4c0b-b277-493dc11061d5 service placement] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:34.563544 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a4bca-f543-4c0b-b277-493dc11061d5 service placement] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:34.623431 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:34.624078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:34.624363 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:34.624643 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] PATH_INFO: `/v3/projects/3ce97bbb75d4412e90231e4da14840bc` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:34.625844 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce97bbb75d4412e90231e4da14840bc)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:34.627767 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:34.766342 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce97bbb75d4412e90231e4da14840bc', 'request_id': 'req-1f6f4b55-8020-4397-9c14-71140df11f99'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:34.773078 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce97bbb75d4412e90231e4da14840bc', 'request_id': 'req-1f6f4b55-8020-4397-9c14-71140df11f99'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:34.779797 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce97bbb75d4412e90231e4da14840bc', 'request_id': 'req-1f6f4b55-8020-4397-9c14-71140df11f99'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:34.861424 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] The token cache is being invalidate because project 3ce97bbb75d4412e90231e4da14840bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:42:34.861715 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:34.862348 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:34.863901 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1430/2895] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:42:34 2022] DELETE /identity/v3/projects/3ce97bbb75d4412e90231e4da14840bc => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:42:34.868203 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:34.869713 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:34.883782 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218b91b-f22b-42a5-aa51-0d6724f66449 admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:34.941511 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:34.942161 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:34.942445 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:34.942725 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] PATH_INFO: `/v3/users/74467350826d43dc869c01ead878842a` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:34.944140 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] RBAC: Authorizing `identity:delete_user(user_id=74467350826d43dc869c01ead878842a)` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:34.946087 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:34.968526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:35.172671 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74467350826d43dc869c01ead878842a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:35.185775 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74467350826d43dc869c01ead878842a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:35.186692 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74467350826d43dc869c01ead878842a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:35.193738 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74467350826d43dc869c01ead878842a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:35.269655 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.notifications [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74467350826d43dc869c01ead878842a'} {{(pid=80317) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:35.276309 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1466/2896] 10.222.0.69 () {58 vars in 1224 bytes} [Thu Jul 28 06:42:34 2022] DELETE /identity/v3/users/74467350826d43dc869c01ead878842a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:42:35.534568 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] Authenticating user token {{(pid=80318) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:35.536011 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.fernet_utils [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80318) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:35.550622 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f4b55-8020-4397-9c14-71140df11f99 admin admin] Validating token access rules against request {{(pid=80318) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:35.609988 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b42a77304cdd4ceda1f777fdc9b5ccf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05bb534ab3954f32aad6d8b3751d3ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80318) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:35.610723 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:35.610922 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:35.611201 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] PATH_INFO: `/v3/projects/240811071c1541a4896bd8c1c14442bf` {{(pid=80318) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:35.612362 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=240811071c1541a4896bd8c1c14442bf)` {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:35.614334 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] RBAC: Authorization granted {{(pid=80318) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:35.641808 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:35.767857 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240811071c1541a4896bd8c1c14442bf', 'request_id': 'req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:35.774604 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240811071c1541a4896bd8c1c14442bf', 'request_id': 'req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:35.781325 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240811071c1541a4896bd8c1c14442bf', 'request_id': 'req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f'} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:35.863986 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] The token cache is being invalidate because project 240811071c1541a4896bd8c1c14442bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80318) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:42:35.864267 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:35.864882 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG keystone.notifications [None req-bf1fe877-b1d1-48b0-adc5-fc8320493f3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80318) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:35.866556 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: [pid: 80318|app: 0|req: 1431/2897] 10.222.0.69 () {58 vars in 1233 bytes} [Thu Jul 28 06:42:35 2022] DELETE /identity/v3/projects/240811071c1541a4896bd8c1c14442bf => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:42:45.272386 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:45.860812 dv-cinder-812429-2-iscsi devstack@keystone.service[80318]: DEBUG dbcounter [-] [80318] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80318) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:26.242150 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] Authenticating user token {{(pid=80317) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:26.243531 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:26.393842 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e3eca-84a3-437f-95e7-c84b5fc0336b admin admin] Validating token access rules against request {{(pid=80317) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:26.455998 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21fb5b-978a-4963-9dca-1f09ba6ffa02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244a8787df7641f2bac9b51213ddb322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a0603e8736d4dfb846b4005972563f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80317) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:26.456626 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21fb5b-978a-4963-9dca-1f09ba6ffa02 service placement] REQUEST_METHOD: `GET` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:26.456921 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21fb5b-978a-4963-9dca-1f09ba6ffa02 service placement] SCRIPT_NAME: `/identity` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:26.457205 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21fb5b-978a-4963-9dca-1f09ba6ffa02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80317) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:26.459049 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.fernet_utils [None req-2f21fb5b-978a-4963-9dca-1f09ba6ffa02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80317) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:26.470929 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21fb5b-978a-4963-9dca-1f09ba6ffa02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:26.472526 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21fb5b-978a-4963-9dca-1f09ba6ffa02 service placement] RBAC: Authorization granted {{(pid=80317) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:26.483416 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: [pid: 80317|app: 0|req: 1467/2898] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:46:36.479834 dv-cinder-812429-2-iscsi devstack@keystone.service[80317]: DEBUG dbcounter [-] [80317] Writing DB stats keystone:SELECT=89 {{(pid=80317) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}